I want to know tester thoughts on what you add and what you don’t add?
4 Likes
regression suite on High impact areas (UI, API) as minimum
Don’t add repeated or similar tests on the same function or scenarios
3 Likes
We add following to our regression test bed:
- Most Used functionality apart from Sanity test cases
- Most impacted areas
Also not to add those which are
- Non-relavant areas for a change (if considered for a particular sprint) until unless you have automation in place.
- As @ Luke mentioned repeated checks on similar scenarios
2 Likes
First, I think you need to think about how much time you have to run through the regression suite. Adjust the number of tests accordingly. There will only be a finite amount of time so you need to be careful what tests you choose to include.
Add a series of tests that cover the most important features in the application. We are not checking that the features are working well, only that they are working.
Next, identify areas of the application that are high risk. These are ones which are used a lot or have changed a lot. Add some additional test cases that cover these features more deeply.
I don’t like to rely purely on scripted testing though. I also like to include some exploratory testing in my regression testing. I usually include both scripted and unscripted tests for the high risk areas. But, remember that exploratory testing is not ad hoc testing. Write down a few notes about what should be covered as a guide. This ensures that key areas are not missed out.
4 Likes
Thanks for all the comments so far… Some thoughts for me to digest there. The level of detail is something that’s on my mind at the moment. I always thought regression should be in significant detail, so if the code gets rewritten in the background or there’s another significant change, then you have the regression to fall back on. I’ve seen it a few times where the detail is missed out of regression and small bugs filter through…
1 Like
The problem with too much detail is that it may take too long to run through the tests. Hence why you need to identify the high risk areas - these are the ones which need higher level of detail.
1 Like
A regression suite shouldn’t be testing every single permutation of a system. You would have covered permutations of areas in user stories, and if you have some fragile code there 1) unit tests should be picking it up and 2) just add that particular scenario into your suite.
Ideally, it should be covering those areas at a high enough level that you have immediate feedback if something isn’t working, but equally isn’t taking ages to run.
It is hard to answer “what do you and don’t you add to a regression suite” because it is ultimately down to your individual system. Everyone here is replying with their company’s software in mind which will differ significantly.
3 Likes
Very true, and this shows that there is no correct answer. It is down to the testers to judge what is best - base their decision on their own knowledge and experience of the software.
1 Like
I’ve got to agree with the folks saying that it depends on the software they’re testing.
I typically aim for the 80/20 rule - the 20% of the software that gets 80% of the use, prioritizing anything that has to happen before my targeted functionality, then the actual functionality I’m after from most important/most used to least important/least used.
Once I’ve got a workable regression test suite up and running, areas that tend to get broken more than others will start getting more targeted tests added to the suite. The idea is that coverage will gradually improve and push the worst problems to the least used parts of the software.
1 Like
Sooo true. There is no defined answer for it. All of us are sharing what we do or what we know! The best suitable options can be considered based on the system under test.
We all QA people have got strong judgement capability (which is needed for our role)! 
@melissafisher - I am sure you will make a suitable choice!
1 Like
Something else worth to consider, apart from the do and don’t.
Define the goal of the regression testing you are thinking to build. Then build it while keep focus on the goal
1 Like
We try to go for covering the key functionality that has to work and then anything around that that has been broken multiple times. We also then have the idea of limited time tests, where we will add something in to the resgression to cover a change that has gone live recently with the knowledge that it will be removed from the regression suite after X months to stop the regression becoming too bloated.
Always find it hard to walk that line between the regressions being too detailed or too general that the scenario you care about being missed, though would generally come down on the more detailed side for a regression as my interest is checking specific things work rather than a more exploratory approach which is saved the actual testing.
2 Likes
Thank you for sharing your thoughts. That’s why I think the goal of the regression suite needs to be clearly defined and reviewed regularly.
The regression suite itself should be designed in a way that makes sense for anyone who look at the first time. Just my personal opinion
We have a lot of code inheritance so we’re primarily looking for unexpected damage from enhancement work. Regression is concentrated around the most frequently used functionality and the functionality that will be most used at the time of year of the release, we also give good coverage to areas that can cause the most disruption if they fail (both for customer and our support desk). We release 3 or 4 times a year and hit code freeze about 4 weeks before release. This gives us just under 2 weeks for regression and then we release internally (product managers, trainers, sales and support) so that they’ve got a couple of weeks for familiarisation and for dev to fix anything they spot before we release the software to our customers. Some of our software is pretty ancient (20 plus years) so experience tells us where we need to concentrate the effort.
Other than the great points above, I find high-level regression checklists quite useful:
- Small quick bullet points / table style list describing the piece of functionality.
- Should include all the core functionality you need to check in order to perform a good regression test.
- It is really useful for skimming over at the end of testing to remind yourself of anything you may have forgotten.
- During a full regression test you can use it more stringently as a checklist if needed.
- I’ve also added a column for automation to remind you that this test is automated and doesn’t need to be done manually. This also gives a good overview of automation coverage.
I find these really useful! I’ve sometimes created expanded versions of the checklist which goes into detail about how to test.
4 Likes