The three legs of the testing stool

Insight Main Image

The following article is a guest post to Zephyr from Rodney West, Senior Consultant and the Editorial Director at Isos Technology. Isos Technology is a partner of Zephyr and is the market leader in solving complex enterprise challenges from Agile adoption and QA to DevOps and they help teams improve quality and speed delivery.

Emphasis on testing is abundant in development theory. Testing frameworks are plentiful. Testing time is a pillar of agile development. There are even entire development methodologies centered on testing. Despite knowing its importance, testing is often only given lip service in product development.

Even when resources are allocated to testing, it is common to only focus on one or two type testing areas. This happens for a variety of reasons, from budgeting concerns to lack of testing expertise. Though some testing is better than none, inadequate testing can easily mask issues.

A stool is a good metaphor for testing. Only using one or two legs will result in a minimally useful, highly unstable stool. Reliability significantly increases when a third leg is added. Testing is similar. At a minimum, projects should incorporate at least three testing areas: unit testing, functional testing, and performance testing.

Unit Testing

Unit testing is the first line of testing. Projects grow in size and complexity over time. New developers roll on and developers with product knowledge roll off. The longer a project lives, the greater the chance for the application to break due to the addition of features and general product updates. Unit testing is key to mitigating issues, saving development time and keeping users happy.

Unfortunately, unit testing is the most ignored phase of development. Why? Many developers view unit testing as a sibling of code documentation. Developers understand why both are important, but they are seen as taking time away from doing “actual work.”

“I’ll do it once everything else is done,” is a common mantra among developers. The problem is there is never a time when everything is done.

From the business perspective, the key advantage of unit testing is it is inexpensive. Additional resources are not required as long as existing developers incorporate writing unit tests into their development process. Frameworks for automating the tests as part of builds are also a strong benefit to good unit tests.

Functional Testing

Where unit testing is the responsibility of developers, functional tests should be kept out of developer hands. Unit testing is a white box requiring intimate knowledge of the code. Functional testing is a black box where the underlying code is magic.

Effective functional testing requires a few process related pieces. First, at least one person involved with the creation of test cases must have a channel to the client, whether directly or through a project manager or business analyst. Second, functional tests require good documentation, including meaningful user stories and descriptive tickets.

An important consideration when coming up with a functional testing strategy is automation. Depending on product complexity, there are a number of testing frameworks that can be used to automate functional testing. The advantage of automation is ease in running regression tests and mitigating tester bias over time. A disadvantage is the person creating tests needs technical knowledge of the automation framework.

From the business perspective, functional testing is the most visible testing. Functional tests are easy to discuss during meetings and allow a project to quickly self-correct. Additionally, if manual tests are used it is simpler to bring on more testers. These testers can be drawn from less technical team members who may not currently have tasks.

Performance Testing

Performance testing is easily the most cumbersome of the three stool legs. Performance testing requires dedicated testing resources, dedicated servers and coordination between developers as well as the infrastructure team. It is arguably the most important part of a production level product. As long as performance is good, people may overlook minor code and functional issues. Good code and good functionality cannot hide poor performance, however. No one wants to spend a week of no sleep tracking down production performance issues that would have been caught with performance test.

Sadly, performance testing is also the hardest sell to the business. Resources, servers and testing time are all expensive. Additionally, when an issue is found during performance testing, it often requires cross-team collaboration to resolve. Is the problem code or a missing database index? Are other virtualized hosts running on the physical hardware running the virtualized performance testing hosts? Is the performance testing environment properly reflective of the production environment or were too many trade-offs made to reduce cost?

Because of the inherent performance testing pitfalls, a detailed strategy should be wrapped around performance testing. Getting business buy-in must be part of this strategy. This will be the hardest part of performance testing, eclipsing technical setup. Once you have this (or your third P1 due to performance issues), a stable performance testing environment will save you development time, save the business money, lead to happier customers and keep you from getting those 2am wake-up calls.

Conclusion

Proper testing is a pillar of development. Testing is most effective when code is properly balanced over multiple testing strategies. Having solid unit, functional and performance testing will go a long way to building stable products, which will create happy users, reduce development time and save money.

Author Rodney West: Rodney West is a Senior Consultant and the Editorial Director at Isos Technology. He has over 17 years of experience in Software Development, Software Architecture and Technical Leadership in a wide range of industries. His approach to software development focuses on the interaction between corporate culture and development processes in the delivery of software.