I’m working on a Scrum team. So in a sprint, we have sprint planning, knowledge sharing, code review… In those sections, we will share our automation strategy with each other. So I’m so familiar with this challenge
1 Like
Below are two automation approaches will be discussed: risk-based testing and the automation pyramid.
Risk-based testing gives higher priority to testing the elements that are most at risk of failing which also carry the greatest negative consequences if said failure occurs.
The financial impact of potential errors
Probability of failure (here it is a good idea to ask the developers what they think)
Service level agreements (SLA)
This should give us a good method for prioritizing which test cases to automate. Another approach is the automation pyramid.
Base Layer: Unit Tests
Clearly in the pyramid, (as a part of the best testing practices for agile teams), most of the testing should take place in the development stage, running unit tests after every build. These tests are the easiest, cheapest, and fastest to complete and are an important aspect of test driven development. Running more tests at a lower level allows us to “check our work” as we go, getting feedback immediately and allowing us to know exactly where the bugs are when it is much harder for them to hide. Here, the bugs will also have a shorter life span, having been born and removed in less than a minute, perhaps. During the UI tests, bugs will have lived for much longer and will put up a greater fight because they have lived there very comfortably for a longer period of time (perhaps even a couple of days).
Mid-layer: API/ Integration/ Component Tests
After we run all of the unit tests and they pass, we can move onto the API/ integration/ component testing phase. Integration tests are run to make sure that all the components work together properly. This is where we can test most of the logic and business processes without going through the UI. It is best to automate here as much as possible. If you have to decide whether to automate at this level or at the UI level, here you’ll have less problems, easier maintenance, faster test execution (meaning finding bugs sooner and decreasing their lifespans) and you get to test the logic of your system. These tests are slower and more complex than unit tests, but they are still faster and less brittle than UI tests.
Top Layer: UI Tests
Last and run least are UI tests. It’s best to run as few as possible as they are costly, more difficult to prepare and maintain, and take a long time. Here you just want to make sure that the user interface itself works properly, knowing that all the other aspects of the system should have already been tested. Automate only the most critical tests, end to end, with a flow starting from the user login and ending with the approval of the invoice. It’s also helpful to focus on things related to the browsers or the UI. Be careful with these tests as they are more likely to provide false negatives and false positives. After running the UI tests, manual and exploratory testing can be conducted (as shown in the sphere shape above the pyramid).
As you can see, the pyramid approach provides a strong testing base in the unit testing phase from which to build upon further testing in the integration and UI phases whereas the ice cream cone approach is more “top heavy” and less stable. To excel in the agile development world, it is imperative to follow the automation testing pyramid in order to produce the best quality software possible.
The animation strategy we undertook was one where we:
- determined the features of highest value worth automating
- worked with Product Owners and Devs to establish proper workflows / critical paths
- coded in the language currently in use by Devs
- integrated with the current build pipeline (inclusive of code reviews)
- presented to the stakeholders a working proof-of-concept that articulateed the solution employed as well as the benefits of having a reliable test harness at work
1 Like
I am afraid that we aren’t currently doing any automation here at the moment, so I’ll have to pass on today! It is mostly something I am learning for future applications.
1 Like
Strategy we are currently following:
- We have a parent automation solution file which is an independent framework with all required capabilities that can be used for any project.
- Our automation suite is powered by the parent framework and is capable to run on cross browser and different mobile platforms.
- This automation suite is currently configured in CI/ CD pipeline enabling the suite to run automatically whenever there is a deployment.
More improvements planned with the team:
- Every feature that is developed should be tested with an mindset on how we are going to automate them
- Automate tests straight after developement
- Developers should be involved in testing both manual and automation
4 Implementing specification by example principles.
1 Like
I have written an article on Agile Test Strategy which can be expalined to stakeholders using a single image.
https://www.linkedin.com/pulse/agile-test-pyramid-risk-identifying-filter-amit-kulkarni/
I’m delayed catching up with the people on Twitter who are participating: