Requirements are still a problem, right?

I was recently writing a paper where I wanted to reference literature that explained how requirements are one of the key failure points in software projects. I was thinking this would be easy – I’d just reference the CHAOS report or some other academic studies.


And intuitively (and experientially), it makes sense to say that all the requirements problems are not solved quite yet! I know I’ve personally seen projects get delayed or deploy the wrong features because of poor requirements. But what surprised me is that I had a really hard time finding recent research indicating scientific results to this end!


Like all of us, I have read the many references to the Standish Group’s CHAOSE report to the top three reasons for project failure being related to requirements. But, I have not read the report myself, as it’s a bit pricey after all. But even if those references are all valid, the CHAOS report is from the mid-90’s (scary thing - that’s over ten years old!). The Standish Group has done updates every few years since 1994, but interestingly, I cannot find many credible sources to validate those CHAOS report updates still reflect that requirements are the top issues.
In my quest to find academic research to support this claim, I once again came up empty-handed on recent research. All of the papers I read as part of my research referenced the exact same set of papers from a long time ago. One paper was from 1976 by Bell and Thayer. It is the generally accepted original work to recognize requirements were an issue (a very interesting read, though clearly not current!). Ralph Young has a nice summary of the existing literature most commonly referenced in Chapter One of Effective Requirements Practices, including work by Boehm in 1981 and Davis in 1995. But again, you’ll be hard pressed to find anything written after the 1990’s.


It’s just very interesting that it seems like requirements are still an issue. And it seems as though the industry and academia accept this as fact. But yet, no one I could find has done a recent study to actually prove this is still the case, or more importantly, to measure what the specific problems are today. In an industry that changes as fast as this one, simply said, it’s really surprised me.

Product Shopping Management

Well the 2007 holiday season is officially underway! And with it, I find myself preparing for the usual hectic shopping experiences that I face every December.

Though today I stopped to think – is it really so stressful for everyone out there? Is it just me? Perhaps that’s it. After all, I know people that LOVE this time of year and the crazy shopping associated with it. Perhaps I just need a new way to look at this situation!

And so begins my attempt to cross-pollinate my way of thinking from work to home - which sometimes works really well and sometimes falls apart. But either way, this is my latest attempt to fit the holiday gift-buying experience into our world of software projects…

Gifting objective

Let me start by saying that I absolutely love to give gifts to my family and friends. However, what I have a hard time with is figuring out what to buy. I want a creative gift for each person. And by creative, I mean something that they don’t already have, something that they wouldn’t just go buy for themselves and something different than prior years’ gifts. I would like it to be something that I like - after all, I’m the gift-giver. And finally, I really would like to enjoy the process of buying and giving the gift.

Release timing

It feels like year after year we are building towards our one big product release at the end of year. And so I must say that the fact we do this crazy-all-out-gift-giving experience at one time per year is a bit nutty to me. I mean, why not just have a culture of randomly buying gifts for family and friends all year around? You could space your personal gift shopping out in quarters or months, whatever – it’d be up to you!

Methodology Choice

I have to say, I know some people that start holiday shopping back in August. What are they thinking? They like to start early and pace themselves, I guess. Frankly, I don’t have the endurance to keep it up for 4 months! I much prefer the agile method of shopping on the fly in December….for everything…for everyone.

Design

Often we play this design-it-yourself game at the holidays:

Family member: “Joy, what do you want for Christmas?”

Me: “I don’t know really. I don’t need anything.”

Family member: “Think about it and let me know.”

(A few weeks pass)

Me: “Ok I found something. I like this sweatshirt.” (I send a link to
something I’ve seen that I like.)

Christmas comes and look at that – the sweatshirt is under the tree.
Surprise!

We are fortunate in that we can really buy ourselves what we want. And the true warmth of giving comes in giving the unique gift that they wouldn’t buy themselves (refer to gifting objective above). So when I have to effectively design the gift I will receive, that’s not fun. And when I have to ask my family to do the same (design their own gift), that’s not fun either.
So I’ve made a decision this year – I’m not asking my family what they want, and I’m not telling anyone what I want. I’m going to completely focus on understanding the basic requirements and then I’m going to have fun buying the gifts.

Success is in the requirements

So like with all projects, requirements are the key to this being a success. I basically need some general family information such as sizes and general preferences such as colors, hobbies, etc. Armed with this data, I intend to head out on my shopping adventure to just select things that meet my gifting objectives!

And with this, I realize I’m doing a bit of product management with my holidays.

Keeping up Appearances

Remember the old adage "You never get a second chance to make a first impression?" I'd like to share my thoughts on that conventional wisdom. I both totally agree with it and think it's crazy. Maybe that makes ME crazy. I'll let you decide for yourself on that one.

In my line of work, making a good first impression is key. I'm a consultant, so I have the good fortune to work with many different clients on many different projects. However, that good fortune brings with it some simple but weighty responsibilities. I have a responsibility to my employer and to my client to make a good first impression. My responsibility to my employer is to make sure that they are represented professionally, and the impression I give to a new client sets the stage for that representation. My responsibility to my client is to provide them with dependable, reliable professional services. A solid first impression ensures them that I and my work will indeed be professional and reliable. Or does it?

How many times have you met someone, had an initial impression of that person, and then only days or weeks later radically altering that impression? I've run into this situation many times in my personal and professional lives. My first feeling is "Wow, I really like this person!," only to realize that I liked the impression, not the person. Conversely, I've met people who, for whatever reason, "rubbed me the wrong way" at first but became a great friend or co-worker over time.

Whether you've had similar experiences or not, you might be saying, "Well, that's great, Mike A, but so what? Things change - we all know that!" Yes, we do all know that, but I think we tend to forget it in our day-to-day interactions with others. The first impression is indeed key, but the ongoing impressions we make on people are equally powerful. While I may not have a chance to make multiple FIRST impressions, my actions certainly can impact others' ONGOING impressions of me, my work, and my company.

It only takes one yawn during a meeting, one roll of the eyes at a comment, one moment's attention paid to email rather than the conversation, or one tardy arrival to a meeting to change someone's impression of you. Then again, it only takes one exemplary requirements model, one case of going above-and-beyond, one smile or "hello" in the hallway, or one artful mediation of a conflict to change that impression, too.

Shouldn't others' impressions of you be based on your work, and not on all this other stuff that has nothing to do with the quality of your requirements documentation? No, it shouldn't. Our work is about much more than delivering an SRS or a set of use cases, and the more you embrace the "intangibles" of requirements engineering, the more successful you will be overall. And whether you're a consultant, an in-house expert, or someone who was thrown into the world of requirements with little or no training, this focus on the impressions you make will serve you and your clients well.

Structure in requirements writing

One of the quickest ways to improve your requirements documentation is to have someone else review your work and provide feedback. Of course, the flip side of asking someone to review your work is offering to review their work in return. And, entire project teams can dramatically raise the quality of their output if every member of the team participates in the review and feedback process, both giving and receiving.


Getting direct feedback on your work from a team member or colleague is valuable but I have found that reviewing the work my team mates have done is also a great way for me to improve my own writing.


After reviewing hundreds (thousands?) of requirements documents, and getting feedback on my own work, I’ve come to the conclusion that the logical structure of the writing is as important as the content. Although it may seem that content is more important than structure, in fact it is the structure that makes a document readable and understandable. If your readers cannot easily comprehend your work the content really doesn’t matter.


Here are some guidelines to use when writing or reviewing requirements documentation:


Build from the bottom up but apply structure from the top down. You will have lots of details and they will need to be grouped and summarized from a very detailed level to a high-level. At each level the groupings should make sense and the horizontal and vertical relationships should be obvious. The top down logical structure of the document should be clear as each level below provides support for the level above. Each level of the structure should be a summary of the level below it.


At each lower level of detail the requirements should be grouped together in a way that makes sense. This may seem obvious but it is all too easy to get lost in the details when writing requirements and just end up with lists of features and functions that are not clearly related.


In most cases the items in each group should be ordered in some logical way. Most often this is sequential or chronological, in the order things will happen, but not always. If a chronological ordering is not appropriate, don’t just list things in random order. Think about how the items in the group are related and list them in that order, by priority, proximity, cause and effect, or whatever.


Of course, the rule of seven, plus or minus two says if you have more than nine items in a group you should consider restructuring.


Try to keep structure in mind when you are writing or reviewing requirements documentation. Content and comprehension are equally important.

Reminder: Use Active Voice

We’re all trying to write easily understood documents. Using active voice helps us achieve this. Jakob Nielsen did a great job of explaining the benefits of using active voice in the first paragraphs of his article Passive Voice Is Redeemed For Web Headings. I recommend giving it a quick read.

The Value of Audits

If the word audit invokes fears of a dark room in front of an IRS auditor, then you're in luck - that's not what I'm going to discuss in this post. Instead I'm going to cover requirements audits and why you should do them. In fact, I'll cover three major benefits of doing requirements audits. Now some of you - if you're still reading - are groaning and saying "audits - just one other non-value added task to place on my already over-loaded plate." Give me a chance to convince you that that's not the case - read on.

Organizational Improvement
The first benefit of doing requirements audits is organizational improvement. Any improvement program/methodology (such as CMMI or TQM) has a measurement component. For example, in the CMMI framework, to get to maturity level 4, you need to quantitatively manage your processes. Conducting audits is a great way to measure how each of your teams/projects is doing against defined objectives. Then to get to maturity level 5, your organization needs to optimize processes - and again, results from audits can feed into that improvement process.

Deliverable Quality
Audits can be performed during logical points during a project, or after the project has ended. If you conduct audits during a project, then the results of that audit can positively impact the quality of requirements. This is similar to having peer reviews performed (see an earlier post on reviews here).

Staff Improvement
Last, but by no means least, audits are a tremendous opportunity for staff development. Most of us get annual performance appraisals, but how many of those appraisals contain significant and detailed information that we can really use to improve our performance? Not many that I've received. Audits are a great way for us to receive measurable and actionable feedback.


I've heard many reasons why people don't do audits - "they are a waste of time", "no one would pay any attention to them anyway, so why bother?", "I don't have time, I'm already too busy", etc... Sometimes it just takes a couple of people to decide they want change for change to occur. Remember, the definition of stupidity is not doing something wrong. It's doing the same thing wrong over and over again while expecting a different result. If you really want to impact your organization, then take some responsibility and try something different.

Hopefully I've peaked your interest and you want to perform an audit. In my next post, I'll give you some ideas on how to plan for and run an audit.

RE is not a Fiefdom

There's a sense of security in knowing what's mine and what's yours. Not so much a sense of ownership (keep your hands off my stuff!), but a sense of responsibility. I know that I'm responsible for mowing my own yard, but not responsible for mowing yours. You know that you're responsible for paying your electric bill, but not mine. These examples and ideas work pretty well for our home lives, but what about our work lives? What happens if we apply the same principles to our work in Requirements Engineering?

It's easy to ride the train of thought from home to work. At work, I'm responsible for the SRS, but not for the database design. I'm responsible for documenting non-functional requirements, but not for user manuals. And I'm responsible for managing requirements change, but not for managing the project budget. Those "other tasks" belong to the DBA/DA, Technical Writer, and Project Manager, not me. Right?

To me, the answer depends on how you look at your project and your role within it. Many project teams (and most teams that I've worked on) have clearly defined roles, and those roles almost often are delineated by job title. A Database Architect does these things, a Requirements Engineer does these things, and a Project Manager does these things. Sometimes the responsibilities are further broken down within each role, so that a team of Requirements Engineers has an administrative lead, a toolset lead, a modeling lead, and so on. This type of division of labor is a good way to break down and structure the work that has to take place on a project.

However, I often see this "division of labor" approach misunderstand (or simply miss) the interconnected nature of our work. The DBA may specify the database platform and normalization rules for the system, which I may then use to help draft some of the non-functional requirements. I may create use cases, and the Technical Writer may use them as a starting point for her or his manuals. The Project Manager may specify the timeframe and budget for the project, which will then constrain the work options for the rest of the team. Our work is interconnected and interdependent. So why do we see our responsibilities as isolated?

I've often referred to the different groups within a project as fiefdoms (another term I've often heard used is "silos"). I prefer the term "fiefdom" because it carries with it a sense of feudal battles over property (my work vs. your work) and a sense of powerless subservience to the king (the project itself). It also conjures up images of filthy peasants toiling in the fields, an image I'm sure we can all empathize with at times, and a way of working that I'd prefer to avoid! fiefdoms, their strictly hierarchical organization, and the "this is mine, that's yours" mentality they support, don't belong in software development - they belong in history books.

So am I suggesting that we shouldn't have clear delineations of responsibilities within a software development project, or that the RE (or the DBA, Technical Writer, Developer, or any other team member) is solely responsible for the project's success or failure? Not at all - as I said earlier, a role-based division of labor is a great way to identify which people will do which work. Additionally, the project will most often succeed or fail as a whole, not because of an individual (although I'm sure exceptions do exist). What I am suggesting, though, is that we shouldn't allow our assigned tasks to limit our responsibilities within and for the project as a whole. Each of us must remember that we are responsible to each other through the interconnectedness of our work and in that way we each carry an individual and a group responsibility for the overall success of the project. We each may not be able to do the work of our peers in other disciplines, but we can and should support them as best we can.

A team, and therefore a project, that is built from a sense of mutual support and responsibility will be more productive, more fun, and more able to adapt to the inevitable changes a project faces. So, leave the exclusionary responsibilities of fiefdoms in the historical muck and work to make your team more fun and successful through shared and supportive responsibility.

Words are hard

To put it bluntly, requirements text is typically pretty bland to read. You don’t get the colorful variety of writing in passive voice, because it can confuse the audience. But also, you have to pick your words very carefully. English is actually quite challenging, in that words have different meanings.

There is a concept called Garden Path Sentences that illustrates this well. And the essence of it demonstrates how people read sentences one word at a time. If something doesn’t make sense in your intuitive first reading, you have to back up and re-read it. The example they give is:

The horse raced past the barn fell.

Technically this is a correct sentence, but it takes two readings to have it make sense.

This certainly something we want to avoid in writing requirements, and so we write in active voice with simple obvious word choices. Bland yes…but also very practical!

The use and misuse of includes and extends

Any sizeable collection of use cases will have relationships between use cases that can be modeled using the UML includes and extends stereotypes. Used properly these model constructs help make the model more readable and maintainable. Used improperly they can make the model incomprehensible and this happens all too often.


The includes relationship occurs when an included use case modifies the behavior of a base use case. The behavior of the included use case is inserted into the behavior of the base use case at an insertion point. An included use case can be included at multiple points in a base use case and can be included in more than one use case. The included use case is specifically referenced at the insertion point(s) in the base use case(s). On a UML use case diagram the relationship is drawn using a dashed line with an arrow head pointing at the included use case. Includes relationships can be nested so a use case can be both a base use case and an included use case. Conceptually, you can think of the base use case ‘calling’ the included use case. Included use cases are often used to encapsulate behavior that is common to multiple base use cases.


The extends relationship also involves one use case modifying the behavior of another use case but the relationship between the use cases is quite different than with the includes relationship. The base use case has extension points but does not specifically identify the extending use case(s). The extending use case adds to the base use case and essentially becomes part of the base use case. Extending use cases are often used to encapsulate alternative and exception flows. In fact, this is a good conceptual way to think about extending use cases. If you have alternatives within alternatives you may have a good reason to separate out the flows into an extending use case. On the UML use case diagram the relationship is drawn using a dashed line with an arrow head pointing from the extending use case to the base use case.


Adding lots of includes and extends to your use case diagram can make it difficult to read so use them only when they improve readability and maintainability, and use them correctly. If you are modeling behavior that is common across use cases, consider the includes relationship. If you have complex alternatives and exceptions, consider using the extends.