Optimizing regression testing in Agile development
Regression testing often becomes a pain point for an Agile project manager. This type of testing presents two major challenges:
- Increased time and efforts. Regression testing may take up a lot of time, especially in large projects. Sometimes, teams have to spend an entire sprint on regression, which is hardly acceptable. Additionally, increased time and efforts mean increased costs.
- Lack of concentration. Re-running same test cases again and again for a long time results in lack of concentration. This can lead to bugs sneaking into production, which also increases overall project costs.
To solve these problems, experienced vendors of software testing services and QA follow certain balanced strategies. Let’s explore some of them.
Regression testing in Agile: How to optimize?
There are three most common ways to optimize regression testing in Agile development:
- Two-level approach
- Test prioritization (risk-based and collaborative approaches)
- Automation approach
Each of them has its pros and cons.
Following this approach, a testing team break regression testing into two cycles:
1. Iteration regression. The team perform iteration regression at the end of each sprint. Iteration regression specifically focuses on features and changes made in the iteration and areas of the application that could be affected.
2. Full regression. Test engineers run full regression before releases and project milestones to ensure that the application works as planned.
However, this approach is not as simple as it seems. To be successful, it requires a seamless communication within the project team. Communicating with developers, the testing team will be able to get a comprehensive view of what was done in the iteration and make a well-grounded choice about the necessary type of regression testing. Another consideration is the time passed since the last full regression. It may be useful to set up a schedule for regular full regression and perform it regardless of the changes made in a particular iteration. This will help the team avoid tedious full regression after every sprint and keep the application relatively bug-free at all times.
Test prioritization: Risk-based approach
Applying the risk-based approach, a testing team choose test cases covering the application areas affected by recent changes in the project and rank them according to the priority, which reduces regression time and efforts.
As the project proceeds, the risk-based approach can be applied to agile regression testing suite. The testing team divide this test suite into three categories:
- High priority. These test cases make around 10% of all regression test cases. They cover the critical functions of the software, areas highly visible to users, defect-prone areas, and areas that underwent many changes.
- Medium priority. These regression test cases (about 30%) describe exceptional conditions (negative test cases, boundary value test cases, etc.). This priority mostly applies to the test cases that repeatedly detected bugs in previous product releases.
High and medium priority test cases are always included in the new agile regression test suite.
- Low priority. These test cases (the remaining 60%) cover the rest of functionality. Testing engineers use them in regression testing before a major release to ensure full test coverage.
Thus, assigning priorities helps reduce time and efforts on regression testing in Agile with no damage to product quality. However, this approach may cause some problems, as the testing team are limited to their own tasks and cannot see the whole picture. Luckily, there is another strategy that fully correlates with the Agile specifics.
Test prioritization: Collaborative approach
This approach allows team members to contribute their knowledge and unique perspective to regression testing. During a sprint, the team work out a regression board covering every piece of work done by every team member. A team member can make slight changes on the board based on his or her knowledge. For instance, a developer understands that a recently added feature may influence the user profile, so he or she adds this information to the board. When it’s time to run regression testing, the team consult the board and assess time investments. Using this approach, the project team apply their unique perspective and knowledge to regression testing, which makes this type of testing more efficient and less time-consuming.
Though reasonable and convenient, prioritization approaches need caution, as they may lead to technical debt, a pitfall undermining overall project implementation.
Warning: Technical debt
Technical debt in Agile development denotes the efforts needed to finish all sorts of incomplete work piled up during a sprint/sprints/project lifecycle. Typically, technical debt appears when the team decide to “fix it sometime later”. And in some cases it makes sense. For instance, if the team postpone security testing until some sprints later when the system is more stable, it goes into technical debt. But it’s a reasonable tactical step dictated by wise time management.
The problems arise when technical debt goes out of control. The Disciplined Agile delivery (DAD) framework offers a whole range of steps to manage technical debt effectively. Surprisingly, continuous agile regression testing is one of them. This brings us back to square one.
So how is it possible to optimize Agile regression testing and at the same time control technical debt? The answer is in regression testing automation, the third optimization strategy.
Automating regression testing in an Agile environment is the best way to deal with most common challenges this type of testing presents. Automated regression testing reduces the testing time from days to hours and spares a testing engineer the tedious task of running the same test cases again and again. Nevertheless, automation of regression testing in Agile should be treated with caution.
The right time is the first thing to consider when planning regression automation. It is better to introduce automated regression testing when the project has already reached some level of maturity, and major changes have already been made. At early stages, most projects undergo changes in the UI, product flow or logic in each iteration, which increases testing efforts (both manual and coded).
Besides, automated regression test suite, however comprehensive, cannot be valid forever. In Agile, automated regression testing demands that the test suite fully reflects the changes and updates in the application functionality. It is reasonable to review it after each major release and discard obsolete test cases that no longer contribute to the product quality assurance.
Still, even with proper auto test suites, manual testing is there to stay. Manual regression testing always goes first, as automated test cases are based on repeatable manual tests that stably detected bugs in the previous iterations. Besides, experts recommend verifying automated regression results with quick manual smoke tests to identify false positives/negatives.
Focus on communication
To optimize agile regression testing, a testing team should ensure seamless communication with other project team members:
- BAs to monitor the changes in requirements and assess them from the testing perspective.
- Developers to learn which changes they made during a particular iteration. This will help balance efforts to assure acceptable quality without excessive testing.
- PMs to update them on the current state of testing and avoid time crunches at the end of a project iteration.
This is a common ground for all the four (or three) strategies for regression testing optimization.
Regression testing in Agile is like a bitter pill: unpleasant but necessary to assure the product quality. Luckily, Agile testing teams have a bunch of strategies they can apply to optimize regression testing. The most popular are:
- Two-level approach
- Test prioritization
- Regression testing automation
Teams usually choose the approach that fits their project best in terms of scale, time frame, number of releases planned, etc. Still, the three approaches have a common ground – a seamless communication.