Can User Stories be used in Waterfall Development?

If you work in the business of software development you most likely have come across User Stories. If you aren’t familiar with them, or are all of a sudden being asked to write them or deliver them and aren’t quite sure what they are, here is a brief background.

User Stories are born from Extreme Programming (XP) and Agile Development Methodologies. XP and Agile development is essentially a set of principles and practices which have emerged over the years (XP was first coined in 1999) to address with the challenges of software development and fulfill a goal of delivering working software as early as possible.

User Stories fit mainly into the requirements gathering and definition phase of software development. They are Agile’s answer to transforming what is often an onerous, expensive, confusing, obscure and inefficient process to one that is lightweight and effective, and drives the software development process towards delivering valuable software features early.

Like many other aspects of Agile Development, User Stories are becoming adopted as an industry standard way of describing software requirements.

The benefits behind writing requirements in this way include:

• Writing requirements as User Stories makes them easy to understand i.e. they eliminate technical or domain jargon 
• Writing requirements as User Stories helps validate their value to the users of the application 
• User Stories can be delivered independently and in short iterations, allowing the ROI from working software to be realised earlier 

However, because User Stories are being appropriated by different organisations who deliver software in less agile and more waterfall ways (wagile, scrumafall, etc) and don’t have all the other Agile practices in place, there is often confusion around what a User Story is and how they can be used to describe, specify and deliver a solution.

In my experience, this is often the case where an Agency or Software House (Delivery Partner) is tasked to define, cost and deliver a new build for a Client.

In this scenario, the following constraints often apply:

• The Client has a maximum budget to get the solution built 
• The Client has a massive wish list of requirements 
• The list of requirements exceed the budget
• The Client wants to know what they are going to get for their budget at the beginning of the project 

Given this starting position, the Delivery Partner has the challenge of defining and delivering a solution which has as many of the items from the wishlist within the budget constraint. So faced with this challenge, can User Stories be used to define and cost a fix priced build?

The short answer, I put forward, is yes with a caveat that User Stories go through the following steps to turn them into a scope of work that can be delivered within a budget.

Step 1: Gather Requirements as User Stories 

In this step, the Client should define a wish list of requirements in terms of User Stories. The Delivery Partner can work with and support the Client on writing User Stories if required, but they should primarily come from the Client to ensure they can be prioritised effectively.

The output from this phase will most likely be a large list of requirements which far exceed the available budget for the solution. This should be apparent even without going through extensive analysis, solution definition and estimation activities.

Warning: There is often the expectation that these User Stories can simply be prioritised and estimated as is. This is definitely not the case and there needs to be successive phases of prioritisation, analysis, refactoring, solution definition and estimation to get to a selection of User Stories that can be delivered within the the budget.

Step 2: Prioritise the User Stories 

In this step, the Delivery Partner should work with the Client to prioritise the User Stories into a MoSCoW list. At a very basic level, the prioritisation is a combination of Business Value (want the business want) and Commercial Reality (what the business can afford).

This is best achieved through collaborative workshops between the Client and the Delivery Partner. As part of this process there needs to be some sanity checking of prioritised User Stories against the Budget. This is generally achieved in the background through thinking through the solution at a high level and high level estimation.

After the prioritisation process, the output should be a reduced list of high level User Stories (or requirements) that both the Client and the Delivery Partner are happy with i.e. the Client is happy with the features included in the solution and the Delivery Partner is happy that they can deliver them within timeframes and budget.

Step 3: Refactor and Add More Detail 

The next step is to refactor the User Stories into User Stories that can be estimated and ideally delivered within iterations of development. The refactoring process can involve breaking down User Stories into smaller stories, grouping stories into a single story, or adding more stories to support new requirements or even non functional requirements. The INVEST Mnemonic describe the qualities that a user story should have.

In addition to refactoring User Stories they will also be embellished with more information such as Acceptance Criteria and Constraints. This will help define the requirements in more detail and help the team plan and estimate in detail what they need to do deliver them.

This best achieved through collaboration and workshops with the Client and internal assessment. The output of this step is a set of well constructed User Story with more depth of definition around them.

Step 4: Estimate and Plan

With a well defined set of User Stories that can be delivered in iterations, the next step is to plan how they will be delivered. This doesn’t need to be written in stone but the goal is to make sure you have a high level plan of what you will be building, when you will be building it and how long you have to build it. Working in iterations or sprints helps team timebox delivery of stories and track progress more easily.

It is also useful at this stage to perform another round of estimation with the delivery team to ensure the scope is still achievable with the team and timeframes.

The output of this step is a Execution phase sprint plan and allotted User Stories per sprint. At this point, there should also some high level solution definition around how it is going to be built. This can include IA, Logical Architecture, Solution Architecture, Data Model, Data Flows, UML Use Cases.

Step 5 : Go forth and Build 

With a plan in place, the next challenge is to deliver the User Stories in iterations. Using Scrum practices such as Sprints helps the team accept, discuss and define what tasks are required to achieve the User Story. In this process, Stories can be given Story Points to track the velocity of the team. Also useful is to add tasks to achieve the story and allocate to team members. Each task should be estimated and updated to Time Remaining to help track the schedule of the project.

The output of this step is, hopefully, working software that meets the client’s requirements that was delivered on time and budget.

Summary 

The home of User Stories is Agile development but they have infiltrated organisations who do more waterfall style development, such as Digital Agencies. This has created some confusion around how they can be used to do up front definition, costing and planning required in waterfall projects.

The challenge with User Stories is to use them to define a scope of work that can be delivered for a fixed price. Although goes against the grain of Agile development they have great value in describing and putting a value on requirements and supporting a better, more collaborative software development process.

Because of this, I believe they should be used even when you have have to do upfront design, scoping and costing. However, to get from a list of requirements defined by the client to a list of stories that has can be delivered in iterations within a fixed budget requires a lot of collaboration, prioritisation, refactoring and defining.