Live from REFSQ'09: Scenarios in the wild: Experiences with a Contextual Requirements Discovery Method

This paper was presented by Norbert Seyff from City University London in the UK. I’ve met Norbert at previous conferences and heard many talks on this topic by him and his colleagues over the years, so I’m getting to see the research evolve over time which is interesting. Often we hear about research in a very early stage with a small set of data, but then nothing else – maybe the rest all die off?

Anyway, they have a tool on a mobile device that allows you to elicit requirements in the field. The tool provides suggested questions to ask and lets you document requirements while being mobile. In this study they took to the Australian Alps to gather requirements for a ski navigation tool in context.

In previous studies, they found it was hard to interact with stakeholder while using the mobile device so it required two people – one to document and one to facilitate the interview. They have now had an improved ART-SCENE CoRE that allows just one analyst to do the work.

Their results included:

• They discovered new requirements that were not discovered in a workshop.
• They discovered fewer business goals than in the workshop
• One analyst was able to walk uphill and gather requirements!!!
• In answer to whether it allows you to generate requirements at a higher rate, the results were inconclusive because they had to shorten their elicitation time.

To the point that they didn’t develop business goal requirements in the field, one participant in the workshop suggested that this may be a result of focusing on system design in the field and Norbert thought that was a reasonable guess. I would need to think through that further – but Norbert added you likely can elicit these business goals ahead in a workshop setting (not onsite); there is no reason to just use this method. Further, a participant in the audience suggested combining this type of work with the concepts I presented in earlier sessions to really understand the business objectives ahead of being on site and prioritize after. So the mobile device idea is really focused on elicitation of detailed requirements but it is not the end-all-answer.

There were interesting observations around working in a field environment:

• There were unexpected and critical delays in using snowshoes to get around!
• The mobile device batteries died in the cold air.
• As a nice take-away from this, if you are going to elicit requirements in the “wild” (i.e. you aren’t sitting at a table in a room together), you really need to know your environment and plan for it.
• Paula Laurent in the audience commented that you really should try to test your environment ahead of time if you are going to do any work in the field.
• Paper-based method didn’t work well because of the gloves they had to wear to stay warm.

I’ve honestly always been a skeptic about the applicability of this research (and I say that with also saying I have the utmost respect for Neil Madden and his student’s at the City University of London!). That said, each time I see this work presented, I’m more intrigued and a slight bit more of a believer.

For most of the work we do day-to-day, I don’t foresee the mobile device helping. Most of our projects which are not meant to be representative of all software projects, are creating requirements for systems where the users’ environment is typically “a desk”. I can pull up a laptop and use my usual tools to do this type of elicitation. That said, I can think of a few times where I was trying to manage a taking notes in a notebook while walking around with users. These situations included things like navigating a catalog call-center going from person to person for a few minutes at each, walking around a semiconductor fab noting how the various tools operate, and finally and most interestingly – getting on to and off of military cargo planes trying to take photos and notes in the small area of a cockpit! In all of these cases, a laptop wasn’t feasible, so I had to resort to paper and pencil, which slowed me down and was still hard to manage writing on – particularly when I had no surface.

So anyway, as it is every time I hear it, the research is very interesting. And it tends to be the case that this group has very engaging presentations which isn’t always the case in this wonderful world of requirements engineering!