Gathering Requirements for Migration Projects (Part 1)

Many IT projects I have written requirements for are focused on migrating existing functionality to a new system. The goal of the project may be to move to a new platform, build a new system from the ground up with the exact same functionality, or to move a piece of functionality from one system to another. Justifications for such projects include moving off an unsupported technology, gaining performance improvements or integrating with other enterprise systems.

The requirements gathering effort for migration projects is notably different than for a new system being built from scratch or for adding new functionality to an existing system. At a high level, the distinctions are in scope definition, understanding original business needs, working with end users, discovering the end-to-end functionality and IT involvement.

Scope:
As with any project, the scope definition on a migration project is critical. The variance in this type of project comes from the specific considerations in scope. Scope discussions should take into account whether to include the following:
• migrating all functionality
• phasing pieces of functionality over a rollout timeframe
• adding or changing functionality
• updating user interfaces

Prior to the requirements phase, a decision should be made regarding whether all functionality is being migrated immediately. If only a piece of a system’s functionality is being migrated, a clear boundary must be drawn around what is in scope for the project. As an example, we worked on a project where inventory management was about 10% of the total functionality in the existing system. This piece was being moved to a new system that was only going to house the inventory functionality. The original system stayed in place, and the inventory management piece was turned off in it. This defined a very clear boundary for the requirements gathering effort.

A joint business and IT decision must be made when considering if any new functionality will be developed. Often, migration projects are driven by IT needs, but the business cannot wait for a new system to be completed before getting additional functionality. The trade-offs have to be weighed, because not adding new functionality may simplify the project allowing an earlier launch.

When working with systems that have user interfaces, there is a choice to whether the changes will be limited to bug fixes, simple style changes or a complete overhaul. This decision should take into account the users’ needs. If you are working with a call center application where minimizing total call time is crucial and the sales reps are skilled in the existing user interface, changes to the interface should be minimal to avoid impacting the reps’ learned efficiencies in the current user interface.

Understanding business needs:
If there is a detailed requirements specification for the original system, a valid question to entertain is whether you can just reuse that specification to build the new system. In theory the answer is “yes”, but in reality, that can be a risky path to take. Even though you are replicating the existing system, you still need to ensure that you are not redeveloping functionality that was incorrectly defined in the first system. To avoid this, start with the original specification, and then interview users to establish how they use the actual functionality.

On the other hand, there may be a temptation to ignore the original specification, even as much as to ignore the existing system, and build the requirements from the ground up. This would require revisiting the true business needs and developing new requirements. If you rebuild the requirements from the ground-up, you will spend more time in gathering and writing the requirements and development will spend more time building them. It actually may be more cost efficient to build what you have and modify it in future releases. In fact, with this approach, it really is no longer a migration project and instead is just building a new system with new functionality to support existing business needs.

Each project will be unique regarding which side of the spectrum is most appropriate. However, somewhere in the middle of these two, there is an approach that uses elements of both to get to the best system in the most cost effective manner.

Working with end users:
As with any requirements gathering effort, you still need to utilize the end users, but in migration projects, there are differences in how you work with them.

In developing new functionality, it is helpful to work with users at a higher-task level to grasp the goals of their activities. And while that view of their role is still essential, it is also important to understand how they actually do the tasks in the existing system. The difference in migration projects is that significantly more of the overall requirements effort will be spent understanding the current system usage.

Similarly, with new functionality, it is often appropriate to discourage the user from focusing on user interface design. While in a migration project, it makes perfect sense to focus on the existing user interface. As an example, the sales rep in the call center application can describe the generic steps to find a product in the catalog, add it to a cart, capture the customer’s information and checkout. However, it is more useful to actually observe the sales rep doing this task and capture exactly what the screen choices are.

If the goal is explicitly to just migrate functionality, then while you should still ask the users “What do you need to be able to accomplish in the system?”, there is a much greater focus on “How do you use the system you have to do it?” and very little focus on “What do you want the system to do that it doesn’t already?”. The unfortunate news here is that the users may not see how this type of effort will benefit them, so it is important that they understand the business objectives behind the project.

I’ve discussed the first three critical PdM activities for a migration product today. Come back Thursday when I will be covering the discovery of end-to-end functionality and the involvement of IT.

Anyone can write a lot...it takes real skill to write a little.

Since our IEEE/SPIN presentation has come and gone, I have been able to start getting caught up on my blog reading. I came across this post from Johanna Rothman that really serves as a great reminder that I personally realize I need on a regular basis. In it, she discusses how she recently had to deal with some magazine editors who decided to shorten a too lengthy article by simply removing the last couple of paragraphs. Rather than simply accept this, Johanna instead sought to apply the 1/3 rule that she attributed to Gerald Weinberg.

For a 1400-word article, I didn't need to cut 1/3 of the pages or 1/3 of the paragraphs; I just needed to could cut 1/3 of the words. And, without much trouble, that's what I did. The article is much tighter, reads more clearly is clearer, and feels "lighter" to me. When I do my 1/3 exercise, it feels a bit like refactoring.

This concept of “dare to pare” is critical when it comes to writing great requirements, but is one that I know from first-hand experience can be difficult to master. Call it a personality flaw if you must, but I have to admit that I love words. Big words, little words, it doesn’t matter - I truly love the written word and the ability to craft the same message in any of an infinite number of ways. I subscribe to two different Word-a-day mailing lists and am genuinely disappointed when the daily missives involve words I already know.

Unfortunately, a desire to play with language can become pathological when it comes to writing requirements. In this realm, our goal should be to establish as much clarity as possible with as few words as possible. I think this makes an excellent adjunct to the disambiguation discussion we had a couple of weeks ago. As much as we need to be careful with our choice of words to avoid ambiguity, we also need to focus a similar amount of energy towards trying to reduce the volume of our words. Why? Two reasons:

1. The more you write, the greater the odds of introducing ambiguity.
2. The more you write, the greater the burdens you place on the consumers of your requirements.

I think the situation with her editors that Johanna relates above is one that happens frequently in the software requirements world too. Given a desire to cut down on the burden of voluminous documentation, many teams will simply look to chop whole sections out of their templates or perhaps buy into the notion of only capturing some of their requirements. What should be done instead is to focus on creating better requirements by applying something similar to the 1/3 rule that Johanna discusses.

Unfortunately, this approach can be difficult for people to adapt to. In a well-intentioned effort to avoid ambiguity, some people will often err on the side of writing a lot to “make sure everything is clear.” When these folks are presented with a pared down version of their requirements it can be disconcerting when they see how much has been cut out. I liken it to when you bring in a professional arborist to work on your trees at home. You look at a tree when they are done and feel like it has been butchered and will never be the same. The reality is that the tree is far healthier since all of the overgrowth has been removed and what remains will now be free to thrive. This analogy carries over quite well to the process of aggressively editing a software requirements document. When this editing is skillfully done, it can drastically increase the signal to noise ratio with the overall effect being that your projects will be able to thrive just like that well-pared tree.

Yahoo releases "design" patterns

We have recently had a number of discussions on the messageboard about requirements patterns. The concept comes from the software design world which uses design patterns to save significant time when architecting software (See Design Patterns: Elements of Reusable Object-Oriented Software )

The concept behind requirements patterns is similar with the thought there are common high level use cases that are repeated over and over in all software. For example, calculating Tax in the U.S. pretty much works the same way in every e-commerce application. Effectively the same requirements have to be rewritten each time. The happy path, the error conditions and alternate scenarios are all similar. However, while the underlying requirements are the same, the actual look and feel could be radically different. There are certainly patterns for shopping carts, purchasing items, manipulating data in tables and so on.

Yahoo recently released a design pattern library. This library is an example of what we mean by requirements patterns. Yahoo calls their library a design pattern library, but it primarily deals with what the user cares about. From our point of view, this means that the library really is a requirements pattern library. Developers could make multiple implementations from the library that would behave the same way, but the architecture of each of those implementations could be radically different. We would like to see a bit more structure and consistency in the presentation though as the patterns library is a mish-mash of use cases, requirements and user interface references. However, this is a promising start to a requirements patterns library.

Requirements Model 3 - Click-Action-Response Tables

My latest requirements model discussion centers around a tool that we developed at Seilevel for a very large and complicated web-based application a few years ago. These tables can be a real life saver when used on the right kind of project.

A click-action-response table can be intimidating at first blush. When you break it down, however, they are simple to fill out and read (a requirement of any good model). Each UI element on a screen gets its own table. There is a description section that provides some simple layout and size requirements as well as any related use cases. There are also conditional display and behavior sections.

The conditional display section shows how the UI element is displayed under various conditions. In the example diagram, there is one button that is used to log the user in and out. When the user is logged in (the 'Logged In' state of the system), the button looks different than when the user is logged out. This section captures those various states and displays.

The conditional behavior section shows how the system responds to user input on that specific UI element under various conditions. In this example, when the user is logged in, the button logs the user out. This section shows how the various states affect the reactions of the system.

These tables are almost like a cross between a state table (discussed earlier) and a use case. Use cases take system state into account in a similar way, but these tables are at a much lower level of detail.

You're probably wondering if anyone could ever complete one of these tables for every UI element on every screen of a large application. Well, let me tell you, they can. We did it. Yes, it is a lot of work, but the UI was very important to the business and it is much less work to hammer out the details here than after development.

The project we used these on was a spec for an online storefront system. Each UI element had different reactions depending on the type of actor using the store. Home users, business users, and government users all had different UIs. The elements also had different displays and behaviors depending on previous choices made by the user. We still used use cases to capture the high-level flows, but these tables ensured no gaps were present in the low-level requirements.

The value you get out of these tables is in the extremely low level of detail you can uncover through their use. There is no other model I know of that breaks down the user interaction at this low level.

Once you get used to using these, they can be turned out very quickly. If the UI isn't that important to the business, they are really too much work. Once again, knowing when to use the right tool is crucial.

The art and science of disambiguation

Jeff Gray reports in his Collection of Ambiguous or Inconsistent/Incomplete Statements that “the 500 words used most in the English language each have an average of 23 different meanings.” I think there are few times in life when you better grasp this fact than when you are either writing software requirements or when you are teaching English to a two-year old. Since I am actively engaged in both of these tasks right now, I thought I would spend a little time looking at the topic of ambiguity.

Mary had a little lamb.

What exactly does this phrase mean to you? Here are some possible answers.

Mary owned a lamb.
Mary gave birth to a small sheep.
Mary ate some mutton.
Mary conned a mild-mannered person.

Far more than being an exercise in silliness, I have found that this example is a fantastic embodiment of the challenge inherent in casually using the English language as an everyday tool. The “Mary had a little lamb” and “Mary conned the trader” heuristics were developed by Gerald Weinberg and Donald Gause in their book Exploring Requirements: Quality Before Design. The authors suggest that writers of requirements must break each statement down into its component parts and explore how the various interpretations of each word may lead the overall meaning to diverge wildly from the original intent.

I find that starting a discussion of ambiguity with this example is a powerful means of grounding the participants in just how pernicious the problem can be. Once people realize that a simple and well-known nursery rhyme can be accurately distorted into something that bears absolutely no resemblance to its former self, it is far easier to convince them that they need to actively hunt down and root out the ambiguity in their software requirements.

Where to start? An excellent resource for someone who is looking for a primer on the topic of ambiguity can be found here – the ambiguity handbook. Written by two computer scientists and a lawyer, the handbook looks at the challenges of ambiguity with a special emphasis on software requirements.

The authors cover a wide range of topics over the course of 80 pages, but there were three sections that I found particularly compelling.

Techniques for dealing with ambiguity - Here, the authors present multiple techniques for dealing with ambiguity during each of three requirements activities - elicitation, documentation, and validation. The establishment of a shared context is identified as a key elicitation technique. During requirements documentation, the authors suggest it is important to work on actively increasing the precision of natural language. Finally, communicating an interpretation of the requirements document back to the original author is suggested as a technique for removing ambiguity when requirements are being validated.

Ambiguous, vague, and uncertain words - This section comprises a large portion of the document and is filled with many examples of words to be wary of along with numerous ways in which each be parsed. A common misuse of “all” is highlighted by showing how the phrase “all persons have a unique national insurance number” can be interpreted two extremely different ways. Does each person have their own unique number, or does everyone share the same number?

Examples - The authors take the lessons discussed previously and illustrate a sample of them using actual ambiguities from real projects. One example is taken from a set of requirements for an electronic toy, the Tamagotchi cyber chicken. If you don’t recall, this toy presented the user with an electronic chicken that needed to be “fed” and “cared for” in order for it to stay alive. One of the requirements reads “the Tamagotchi dies when it is not fed for one day.” What exactly does this mean? Will the electronic chicken “die” when it is not “fed” for one day, or will the toy itself cease to function if it isn’t given new batteries daily? Probably both, but that is a different issue altogether.

Of course, the reality is that ambiguity is all around us and always will be. I know for a fact that some of you will read this entry and have a few chuckles as you count the number of ambiguities that this “article about ambiguity” actually contains. Unfortunately, the cost-benefit analysis for removing all possible ambiguity from this blog post does not support me taking the time to do so. Can the same be said of your software requirements? If you think the answer is no, then spending some time learning about the research around ambiguity will likely pay off in the long term.

Traditional software requirements don't work?

Recently I have talked to a number of clients that have referred to the fact that traditional requirements don't work. It didn't take long to ferret out that they were referring to creating software requirements using a waterfall model. I want to state for the record that I completely agree that the waterfall model doesn't work. However I disagree with the statement that traditional requirements don't work.
If you look at pretty much any software engineering book, you will find descriptions of multiple software development models including iterative, spiral, evolutionary, modified waterfall and waterfall. There are many others but they are often just modifications of the iterative model. All of these models are old enough to be "traditional" models. Interestingly, all of the models essentially break down to iterative vs. waterfall and all of the methods (except maybe extreme programming) involve "traditional requirements."
The variations of iterative models for the most part just change the duration of the iterations. For example, Extreme Programming has iterations of a few weeks and Agile has iterations of a few months. The key point is that iterative models are just as "traditional" as the waterfall model and have been around for at least a few decades.
So what model do we advocate? Like most everyone these days we prefer an iterative approach with full development phases taking from 3-6 months (we work on software projects of 20+ software developers). Within these 3-6 month phases we use a modified waterfall approach. Design starts as soon as enough requirements are completed and development starts as soon as enough design is completed. We typically sign off on a requirements document that still has outstanding issues, but that has the biggest risk areas resolved. We will continue to change the requirements as necessary, but via change requests to ensure that all the stakeholders are aware of the changes.
So to those who ask the question "do traditional requirements work?" I would answer resoundingly yes.

Two weeks from tonight - Seilevel presents

On Wednesday evening, February 15th, Joe Shideler and I will be presenting at a special combined meeting of IEEE Computer Society, Austin Chapter and Austin Software Process Improvement Network (A-SPIN). The title of our presentation is Beyond The System Shall - A Journey From Good to Great Requirements. This is the event that was originally scheduled back in December but was cancelled due to Austin's blizzard of 2005. Here is the overview:
Today, only a small percent of IT projects succeed while the rest significantly under perform or fail outright. Seilevel will begin the presentation by delving into the statistics behind this statement and then move on to discuss how poor software requirements are a primary reason. The presenters will explore the root causes behind the problem and then provide suggestions as to how companies can improve the situation. A key insight is the notion that requirements engineering will never be a one-size-fits-all discipline. Success requires that companies not only find the right people and give them the appropriate resources, but also allow them to develop and apply a varied set of tools for meeting disparate software requirements challenges. Of these tools, perhaps none are more important than the set of models that can drive completeness and clarity into software requirements specifications. The presentation will close with an overview of these models including examples of use cases, state diagrams, decision trees, and others.If you have been enjoying the Requirements Models blog entries that Joe has been posting, here is your opportunity to see him discuss the first two models in person (model1, model2) while also getting an introduction to some of the other models he will be writing about over the coming months.
We will be starting at 6:30 and Seilevel will be providing pizza and a door prize.
More information and directions to the event can be found here.
See you there!