Agile and Lean thinking changed the way we develop software. That’s for sure. But I think it didn’t change the way we implement software so much that it changed the world around it. For example testing.
There’s a lot of good blogs about Agile testing but the article in Wikipedia at the moment is in my opinion too short to provide encyclopedic coverage of a subject. I think it reflects how straight-forward it is to discuss about how to introduce Agile values and principles to testing. But how hard it is to find practical commonalities that would fit all. This is yet another experience sharing about test strategy and how to organize testing.
The Best Test Strategies Emerge from Self-Organizing Teams
Agile principles state that the best architectures, requirements, and designs emerge from self-organizing teams. I would like to expand this to Test Strategy.
In the waterfall world, I always questioned people that claim that you should “test as early as possible”. In waterfall, as early means as low level tests as possible. When trying to cover all possible scenarios that would happen on integrated SW and HW e.g. in high load situations the low level test suites become large and complex and hard to maintain.
Old-school Test Strategy defines in which test phase different types of tests should be done, what tools should be used etc. There is exit and entry criterion that harden the waterfall thinking. Test Strategy is often made by formal test leader that tries to optimize the test-chain up-front. It leads to over-simplification that is hard to take into practice when the features and functionalities change.
The Best Test Strategies Emerge from Cross-Functional Self-Organizing Teams. Having all** test activities done in cross-functional teams, they have the authority to define the test strategy: They have the possibility to choose in which test environment each test activity is executed. There is no exit/entry criterion since there is no handovers. Testing as early as possible becomes meaningless. Testing as efficiently as possible takes over.
Do we have a test strategy that is different for each team? No. Self-organizing teams will sub-optimize, but sub-optimization leads to the famous “optimizing the whole” when the teams collaborate. The best architectures and the best test strategies will emerge only if the teams collaborate.
**The Separate Test Team Problem
Should we have all test activities in the cross-functional team? Delivery of working software frequently in Agile principles translates to potentially shippable software after each sprint in Scrum. In the by-the-book Scrum this means that we should have all test activities in Scrum teams. I think in the area of embedded software we have to make compromises. I would not put an Airbus test pilot into team developing cockpit software. And I would not run the test flight after each iteration in each team.
In our context, having all test activities in a Scrum team is in conflict with the basic definition of a team. We have test activities that ensure the non-functional properties and characteristics of the whole product. These cannot be connected directly to any feature or any user story. These tests are sometimes needed even without any implementation changes. We don’t have the interdependency between team members here. We don’t have the synergies nor shared goals.
There is also other reasons to exclude some test activities from Scrum teams e.g. specialized competence, expensive test environment etc. But I think these can be fixed. The team aspect we cant.
At the end, we have to make sure that the Test Teams are following scope-wise the Sprint pulse and re-basing the software from the Continuous Integration flow. We should avoid mini-waterfall. Test Teams should be just as any team but with their own specialization. Together with the Scrum teams, the Test Teams should be responsible of having potentially shippable product after each sprint.