How Behavior Driven Development Can Fuel Your Software Testing Program
What is Behavior-Driven Development?
The software development industry is flooded today with ‘new ideas’ that are supposed to help make things better. Agile. Scrum. Test-Driven Development. RAD. Many of these ideas are valuable and can improve your development and testing process. With a market flooded with ideas, how do you decide where to start? Many of these are complicated to implement, call for retooling and culture change, and often the methods to implement are unclear. What’s so special about Behavior-Driven development (BDD), and why is it worth your time
Behavior-Driven Development is, conceptually, a derivation of Test-Driven Development. BDD was developed by Dan North, and it has been around since the mid-2000s. Its goal is to bring the most important code and test cases to focus. It affects how you write requirements, how you write code, how you write test cases, and how you test code. It also is designed to help bridge the gap between non-technical people, attempting to write down what they need, and technical people, attempting to build what the non-technical people asked for.
All these promises sound like they require massive changes to achieve. Let’s walk through an example and see how big the impact will be to your processes.
How Behavior-Driven Development works
Implementing BDD is comprised of the following steps:
- Change the structure of the sentences you use to write requirements to follow a particular pattern
- Focus writing code to fulfill the criteria of the sentences
- Focus writing test cases on testing that the criteria of the sentence is met
- Rejoice that things are more clear
Sounds too good to be true, doesn’t it?
In Behavior-Driven Development, you change your requirements to look more like this:
Title: Give the story a clear title
Narrative: write a simple paragraph that states:
- Who is the primary user
- What is the effect the user wants
- What is the business value for the user
Write as many acceptance criteria as you need to describe exactly what needs to happen to satisfy the user. When you write them, write them in the following format:
- Given <this situation>
- When <the user does this>
- Then <this outcome occurs>
That’s primary change called for in what you do today. By changing how you think about acceptance criteria and writing them in this format, you construct how you state your problems more like how the developer solves the problem in code, and more like how the QA tester builds test cases. The most likely way to solve the problem is obvious, and the way the product must work is also more obvious.
Software development and Testing gain the immediate ability to identify and focus on what code and tests must be built first and are most important. Developers and QA can also flag their code and tests with direct references to the section of each acceptance criteria it addresses by adding comments that copy and paste from the acceptance criteria. Now your code and test cases are searchable based on each requirement and section of requirement. You have direct traceability from requirement to code to test case, which reduces cost for future changes and for defect resolution.
A typical Agile Shop
Let’s walk through an example at a typical Agile Development shop. The shop is using off-the-shelf software to run their Agile process. They have the typical roles- Business Analyst, Developer, and Quality Assurance. A ‘Green’ initiative in the company has led to more effort to recycle consumables, reconditioning them and returning them to inventory, rather than simply throw them all away. The items can be recycled once before they go bad and must be thrown away permanently. The system needs new features to support this new initiative. The software development team is called together, holds a meeting with the product owner, listens to the software problem, talks it over, and then sets about their tasks.
First, the business analyst writes up the user story. He creates the story itself:
As an Inventory Manager, I want to be able to add recycled items back into inventory, so that I can reuse them.
Next, he outlines the functional needs:
- A method is needed to add recycled items back into inventory to be used again.
- A method is needed to identify recycled inventory so that it is not reused again.
After that, he outlines some business rules:
- Consumable inventory items may be recycled and added back into inventory.
- A recycled item may only be added back once. It must be disposed of after the second use.
- Non-consumable inventory is not recycled and is not considered for this story.
Next, he outlines some acceptance criteria:
- There must be a way to add consumable inventory back into inventory as a recycled item.
- There must be a way to identify recycled consumable items in inventory.
- Recycled inventory, once removed to be used, may not be added back into inventory.
- Non-consumable inventory may not be added back as recycled inventory.
The developer now looks through this documentation. It accurately describes the problem, but the developer is left with where to start. She decides to start with data first. The system already has flags for consumable inventory and permanent inventory, so she adds a flag for recycled and modifies the user interface to handle the new flag.
Next, she starts designing changes to the add inventory function. She adds the ability for the user to indicate if a consumable being added to inventory is a recycled item or not. She adds the ability to indicate how many times the item has been recycled. She adds error messages if the item is recycled more than once. She adds error handling to prevent a non-consumable from being flagged as recycled, even though it shouldn’t be possible, to make sure her code handles all the acceptance criteria cleanly and has good coverage of possible permutations. All this ends up affecting a lot of screens, as the recycled flag must be handled wherever a user can add, view, edit or delete inventory.
The QA Analyst looks through the story and builds test cases, including:
- Trying to add a recycled item
- Trying to add a recycled non-consumable
- Trying to add a recycled item back into inventory
- Trying to remove a recycled item from inventory
- Trying to change a consumable into a recycled item in inventory edit screens
The team completes the code in a few weeks. It goes to production and things are fine for a while. Four months later, a lot of new features have been added that depend on the recycled product feature. A bug shows up that somehow allows recycled items to be added back into inventory as a regular consumable. QA looks through their test cases and cannot find a test case for that. The Business Analyst can’t find the business rule specific to that. Development starts picking through code. Everyone on the team understands the problem and knows it shouldn’t happen, but they aren’t sure why they made the design decisions they made. Eventually, it’s traced to a problem with the recycled flag. No one is sure why they chose to go with a flag, but it meets the requirements and test cases. The BA documentation, the QA test cases, nor the code comments line up cleanly, so it’s hard to say exactly what went wrong with their logic process originally. On top of that, the lack of line-up of everything makes it difficult to identify all of the implications of changing how the flag behaves. The group hunkers down for a round of analysis, repeating old research, to solve the problem.
The Business Analyst may very well go on and develop all of the other documentation that happened at the Agile shop – calling out a list of business rules, for example, or functional requirements. Under BDD, however, the Acceptance Criteria is the key driver to what everyone does. When the Business Analyst has finished work he passes the card for the story on to the Developer. When the Developer opens the card, the acceptance criteria make the desired end result obvious. It can be read as a single, clear sentence, but note what breaking it up into lines does:
- Each part of what you want to happen is discrete and easy to understand
- Each line effectively translates to a single chunk of code
- Each line effectively translates into a testable criteria, so writing test cases is easy
She looks at the user’s two steps: the user enters a product number. The product is not a recycled product. The developer modifies adding new product numbers to inventory so that when a new consumable type is created, a second product number is also created that ends in –R. Inventory that ends in –R is recycled inventory. She adds code comments “the system adds the consumable to inventory as a recycled product” and “the product is NOT a non-consumable product”.
A BDD Shop
In a software shop across town, an Agile shop has implemented Behavior-Driven Development. They are still using the same tools as the other shop, but they now use BDD to drive how they write requirements.
When the same requirement comes up for recycled items in their shop, they meet, discuss requirements, and the Business Analyst begins by writing the following:
First, he gives the story a title.
Title: Add Recycled Products to Inventory
Next, he writes a simple narrative of what’s trying to be solved.
Narrative: Users need a method to add recycled consumables back into inventory. Products may only be recycled once. Users need to be able to tell a recycled item from a regular item.
Next, the business analyst writes the acceptance criteria in BDD format:
Given that a user needs to add a consumable to inventory when the user enters the product number and the product is NOT a recycled product and the product is NOT a non-consumable product then the system adds the consumable to inventory to as a recycled product.
She next modifies the add consumables feature so that the user is to enter the current product number and the number to be added to inventory. If the current product number doesn’t end in –R, then the inventory is added to the –R product number. Now it’s recycled inventory. She again adds code comments “the system adds the consumable to inventory as a recycled product”.
If the current product number ends in –R, then the system will know it’s recycled inventory, and gives the user an error, telling them that recycled inventory cannot be used again, and giving instructions for disposal of the inventory. She adds comments “The product is NOT a recycled product”.
The QA Analyst builds the following test case immediately:
- Try to add inventory to system. Enter product number that is not for a recycled product.
- Try to add inventory to system. Enter product number that is for a non-consumable product.
- Try to add inventory to system. Enter product number that is for a recycled product.
He adds to the first test case the comment “the user enters a product number and the product is NOT a recycled product”. To the second, he adds the comment “and the product is NOT a non-consumable product”. To the third, he adds “then the system adds the consumable to inventory as a recycled product”.
Suddenly, your Business Analysts, Developers and Quality Assurance all speak the same language. There are many things you can test about the code above, but the most important test is clearly the one that aligns with Criteria 1. If your solution doesn’t meet this test, you shouldn’t release. Getting from development complete to tested and production-ready becomes a much shorter path. If a bug appears four months later for this team, they identify what the bug applies to, for example, adding recycled products, and they can immediately do a text search and find the acceptance criteria for it, the code for it, and the test cases. It takes minutes to achieve traceability and start diagnosing the problem.
Note also how the construction of the acceptance criteria leads both the developer and the QA analyst to different solutions than before. The developer constructs their solution differently, that takes on the problem directly, and is likely cheaper to build. The QA analyst immediately jumps to the most important test cases. They can see what the must-do criteria are immediately. They will write additional test cases, of course, but the key tests stand out.
Implementation of Behavior-Driven Development is straightforward. First, there’s a period of training for your people involved in your requirements documentation- whether you’re doing Agile, Waterfall, or something else, this approach will work. It’s a simple redefinition of how you write acceptance criteria, not a radical alteration to process.
Second, you must communicate the changes to your developers and QA teams. Work with them to educate them on what the change will be, and how to read and interpret the new method of documentation. Plan how you will incorporate this into your code and test case documentation. You don’t want to ruin the traceability that BDD offers by failing to set standards around how you will document it.
After that, you set a date, and start. It’s best to start with a single project and single release, to pilot the process first. Establish some metrics before you start- were requirements faster or slower to document? What about code? Testing? Were defects reduced? Explore your existing metrics and set up some measurable to make sure that the changes work for you.
Another attractive aspect of this is that if it somehow fails to meet your needs, it’s easy to undo. You go back to writing requirements the old way. No retooling. No big process rewrites. No retraining. Just write sentences the way you used to do it. Both the cost and risk is very low with BDD compared to many other methodologies out there.
Adopting Behavior-Driven Development is very straightforward. It calls for changes to how requirements are written and holding some educational meetings with your teams, but it does not mean retooling, major process change, or culture shift. Risk of implementing is very low, since it can be removed very easily. Behavior-Driven Development is a worthwhile practice for any software shop to at least evaluate and try, and it has the potential to have big benefits to your development and software