Let's make a deal!

Joel Spolsky writes here about the penchant that many organizations have for playing Monty Hall when it comes to the fine art of requirement prioritization. You're likely already familiar with how this exercise gets done in most organizations:

Step 1: Assemble a cross-functional group of the top big-wigs and engage in some serious horsetrading around whose pet requirement is a must-have versus whose is a nice-to-have.

Step2: Lather. Rinse. Repeat.

The end result, oftentimes, is a set of requirements that is more representative of who in the organization carries the bigger stick rather than an accurate prioritization of the features that will most efficiently meet the needs of the business. Lots of pretty bells and whistles - Marketing runs the house. Ten different features that are each required to close a "major deal" - the Sales team is at the wheel. A huge refactoring effort that will have zero visibility to the end users - perhaps the founder is still the CTO.

Joel takes this process, as he often does with SOP in the industry, and turns it on its head. He suggests an immensely rational approach that does away with prioritization by fiat and replaces it with a market-driven framework based on economics. His words are far better than mine so I will just urge you to take a look at what he advocates.

There are two groups in particular who I can see coming back and saying this approach would never work for them.

The first is comprised of folks that work for large companies, particularly people working in IT for large companies. Their immediate response may be that this process would never scale from Joel's 20 person cohort to a large organization with oodles and oodles of diverse stakeholders. Perhaps the exact mechanics of getting everyone in a room - yes. But, I would suggest that it could still work quite nicely if only someone in the company would write up a quick little web application that would allow you to do all of the same simple activities with a large group of distributed people.

The second group of likely naysayers are those people in organizations that are very top-down driven. "Our management would never cede that kind of control to the user rabble." I see a couple of possible ways around this issue. My preferred method would be to use the web-based approach suggested in the last paragraph but set it up so that you could aggregate the input from management separate from that of everyone else. If management and everyone else agrees then everything is great. If not, you have some dispassionate numbers that can be used to drive a discussion around why management's decisions should override the thinking of the many. Alternatively, you could use some form of weighting approach that would allow you to factor management's input at a higher rate than everyone else. I have no idea whether it should it be 2 to 1, or 10 to 1, or 100 to 1, but I imagine in most organizations it would be quickly evident.

As Joel says, "it's not perfect." Even so, I think it is a fantastic start for organizations trying to get off the treadmill where bad priorities lead to even worse products. If nothing else, this approach to prioritizing requirements should lead you at least a little closer to the systems your users want and need rather than those that resemble old Monty's goat!

"Shall" we require?

Does anyone know the history of the shall statement in requirements?

I hear people now and then complain about that word. I personaly don't like it as it's not a commonly used word in English. To that point, a client this week pointed out that when sending requirements off-shore to developers that do not speak English as a first language, it is important that requirements are written in basic English (think elementary/middle school). The requirements need to be written using words commonly taught in English language classes, and "shall" is probably not one of those words. Granted the other side of the argument was that if every requirement says "shall", the developer is probably going to figure out quickly what it means.

So I suggest, why not use "will" or "must" instead?

But that all aside, I'm curious, what is the history of using "shall" in requirements? A quick search did not land me with any obvious answers to it so far.

Who Wants to Write Requirements?

Recently, a colleague passed along a piece by Karl Wiegers. I have been meaning to do a deep dive into his website for a while now and So You Want to Be a Requirements Analyst seemed like a great place to start. I was not disappointed as Karl does an excellent job of painting an accurate portrait of the requirements expert and the project impact she can wield.

Straddling the gulf between vague customer ideas and the clear specifications that will guide the software team’s work, the analyst must first understand the users’ goals for the new system and then define functional and quality requirements that allow project managers to estimate, developers to design and build, and testers to verify the product.

I have often witnessed the multiplier effect that a Product Manager can have on a project and this is most certainly due to the interconnectedness that Karl summarizes so well. Later in the article he highlights some of the research by Barry Boehm that actually quantifies the effect an experienced and skilled Product Manager can have on a project.

...seasoned analysts can reduce a project’s required effort by one third compared to similar projects with inexperienced analysts, and projects with highly skilled analysts require half the effort of those using the least capable analysts.

The aspect of the article that I found most compelling was the manner in which Karl captured the myriad tasks that must be performend by the person responsible for requirements on a project. In particular, I thought his description of the elicitation and analysis aspect of the Product Manager's job was right on the money.

A proactive analyst helps users articulate the system capabilities they need to meet their business objectives...Look for derived requirements that are a logical consequence of the customers’ requests, as well as hunting for those implicit requirements that they expect but haven’t verbalized. Spot the vague, weak words that cause ambiguity and confusion. Point out conflicting requirements and areas that need more detail.

Far too many organizations, in my opinion, look at requirements as fruit that must simply be picked from the trees in the user community orchard. In reality, gathering requirements is a fairly trivial exercise. The real challenge lies in first eliciting requirements from users that typically have very little experience in expressing their needs. After all, most users have unfortunately never been asked to contribute to the development of the systems they have to use every day. Once a candidate pool of user requests has been elicited, the challenge becomes analyzing those requirements to ensure that they represent a complete and accurate depiction of the solution to be developed.

Although I liked the article overall, there were a couple of points where I found myself disagreeing with how Karl presented a topic.

The first, early in the article, was where he stated "Requirements analyst is a project role, not necessarily a job title." Though he goes on to quantify the positive impact that a seasoned analyst can have on a project, I wish Karl had been more forceful in stating that the requirements function should be a titled and dedicated role on projects. Almost anyone would scoff at the notion that software engineer should be just another role on a project so why should we accept any hint of a suggestion that working with requirements is a task to be simply parceled out among fungible work units (a.k.a. project team members). Of course, here requirements experts find themselves fighting for recognition of their unique skills along with other interchangeable (ha!) project roles like QA and technical writing.

Towards the end of his paper, Karl discusses the role that experience and knowledge in a given business domain should play for those working with requirements. Perhaps this doesn't quite qualify as the third rail in Product Management circles, but it certainly is a controversial topic that can inspire heated discussion. Roger Cauvin has a post where he expresses the viewpoint that "experience in a particular industry is little more than a crutch" for Product Managers who might lack the facilitation and analytical skills that represent the core competencies of the profession.

Though Karl certainly seems to pay respect to the role played by requirments expertise, I think he gives domain knowledge a little too much credit by terming it a "powerful asset." Yes, domain experience is useful and has its place. That said, I firmly believe that the value added by a good Product Manager in a domain where he has zero experience will far outweigh the likely harm that most business domain experts would inflict on a project were they to be annointed the requirements experts.

Want to know what to build? You just have to ask...

Asking effective questions is undoubtedly one of the most critical skills needed by anyone involved in writing requirements. One-on-one it's usually called interviewing while group sessions typically involve facilitation. Regardless of the label you apply to it, the simple act of asking a question is actually anything but.

Esther Derby wrote an excellent piece that can be found on the AYE site - Building a Requirements Foundation Through Customer Interviews. The article is a helpful primer on the art of the question. Esther certainly nails the crux of the matter when she states up front that "building the right product starts with asking the right questions." She goes on to describe the different types of questions and the role each can play for Product Managers. Perhaps her most useful piece of advice is the suggestion to have a dedicated scribe assist during interviews and elicitation sessions. At least for me, I know the process of trying to take accurate notes always interferes with the rhythm I want to establish and maintain during a good Q&A session.

Let's face it, some people are born communicators while others are not. That being said, verbal communication is not purely a gift but rather is yet another skill that can be learned and improved with practice. Understanding the techniques of effective interviewers is the first step and I believe that Esther provides a good starting point here. Beyond that, her suggestion to practice with a co-worker is one I have followed in the past with extremely positive results. Watch out Mike Wallace!

Requirements of a Smoke Ring

Want to hire great Product Managers?

The best approach I have seen for evaluating almost any potential hire is to use an interview format that is based on a real world audition. I believe this is particularly true for a role like Product Manager where the hiring manager is faced with the overwhelming challenge of locating candidates that possess deep skills across a wide variety of domains.

Johanna Rothman has an excellent series of posts that describe this interview technique. Here, here, and here are some of the better articles. Her primary thesis is that hiring decisions should be based on solid evidence that the candidate is the best fit for the position. Just as actors prove they're right for a role by trying out, technical hires should be able to provide similar proof through an audition process.

This is an approach that we apply quite rigorously here at Seilevel. I have personally interviewed at dozens of companies, including some that are world-renowned for their stringent hiring practices, and I must say that Seilevel's process ranks at the very top for its comprehensive nature. We have 7 separate auditions that take place during 5 sessions. Each of these, in turn, is measuring the candidate across multiple dimensions. All of them, without fail, are directly evaluating skills that we feel are critical requirements for the people we hire to be Product Managers. We aren't talking brainteasers here, though Johanna might try to apply that label to one of our auditions, but rather a variety of exercises that measure a candidate's abilities under a mix of real world conditions.

The downside to the audition approach is that it can be quite expensive. It takes an upfront investment of time to develop an effective set of auditions. After that, auditions tend to require somewhat more effort on the part of interviewers than the "couple of questions" approach that many companies currently utilize. The upside to this technique is a much greater level of confidence as to whether a candidate actually has the right skills for the job.

Is this approach perfect? Of course not. Any technique short of hiring candidates for some sort of lengthy probationary period will always involve a degree of artificiality that will lead to both false positives and false negatives with regards to hiring decisions.

Is this approach worth it? For almost every position I would posit that hiring decisions that consider some type of audition will prove right far more often than they are wrong. In most organizations all it takes is the avoidance of a single hiring mistake to provide a positive ROI for an audition based approach. You do the math.

Should you consider applying the same level of rigor for your Product Manager auditions as Seilevel does? The answer to that question is actually another question. How do you perceive the role of Product Manager? If you see a Product Manager as being "just another member of the team" then it would likely not be worth the extra effort to make your hiring filter as fine-meshed as ours. If, however, you view the Product Manager as a pivotal role on the team - a person who can exert a multiplier effect on the ability of the rest of the team to develop the best software possible - then the extra investment is definitely worth it. We have certainly bought into the latter viewpoint and the resultant approach has proven its benefit for both our company as well as the clients we serve.