Reaching out to all you test freaks out there
I’m currently working as a test manager for a big international bank in Brussels and I’m responsible for a banking application both on the web and on mobile. We are in the transition between manual testing and automation testing but we have a very large regression pack to execute and every time it’s a big frustration for the testers to start doing this. There is not a lot of new stuff to learn and it’s highly repetitive and boring.
Automation is certainly helping them to reduce the regression pack but we have other challenges there ( not a lot of microservices or API’s so a lot of UI testing still needed).
I’m looking for ways to keep the team motivated during regression testing and even make them eager to start doing it … I’m already trying to move them to a more exploratory based approach mixed with risk based testing to decide what area’s need regression.
But any idea’s out there ?
Thx, my team would be very grateful
Reaching out to all you test freaks out there
Exploratory and risk-based testing is a great way to break the monotony, so that is a good idea.
Back in the distant past, when one of my teams first broke the regression-testing-test-case-factory cycle, That is, more or less, exactly what we did.
We arranged our (not automated) testing activities into categories or features. We gave the testers a feature to test, a checklist of things which simply MUST be tested, and a time frame to finish the test. We had to deal with slow reacting hardware, so this was sometimes days, but for most contexts, hours would do. Then our testers would test the features in any way they wanted.
Once the testing was done, we talked wiith some key players about what we tested, how we tested it, and what potential issues we found. During the end of testing debriefs, we could also talk about coverage, how we could have better tested, if we could automate certain parts of the testing (I was, and still am, a big fan of writing scripts), and so on.
For us, the testers, it was not only a big improvement over the test-case based regressions, but we started finding new issues which would never have been found in the other way.
I’m keen to tackle this as well. We do a lot of regression testing at my work and it definitely gets boring. We use test scripts for it, but, my approach is that they should be more of a guide/reference to try and encourage exploratory testing. I tend to say that we should run through the basic workflows without ticking things off the test scripts but to just refer back to them to make sure we’re not missing anything. My hope is that it gives us a bit more autonomy and the room to do exploratory testing. We don’t work on things as stringent as banking software, though, so we might have a bit more freedom to “play around” a bit more. This may all change, though, as test script results are going to be used as metrics for the team so we’ll see how it goes when that kicks in properly.
Longer term, we’re going to be looking to automate regression testing as much as we can so we can focus more on exploring.
@brian_seg yeah that’s more or less the goal for us as well. Problem is that due to the very hard to script automation ( stuff like authentication and signatures cannot be bypassed and are on hardware ( connected card reader ) platform ). We found some solutions for the authentication and signature. But all UI scripting is slow and prone to error and way too much maintenance. In general even with the risk based approach on a big release we have around 600-700 regression test cases to be executed manually. So needles to say it takes time . Good thing is we find bugs ( read good for the spirit of the testers not so much for a first time right )
I like your idea of the testing debfriefs, I’m going to see how I can implement that in the run. Something like a weekly thing or so. Thx for the tip.
@ollyong_johnson, exploratory testing is a great way of tackling stuff fast. Only downside in my book is that you need good experienced testers to find bug fast in that way of testing. They now area’s of the software that are prone to cause bugs and will focus on those. We always go through all the basic happy flows of the software even if we are convinced that the soft was not touched in that area. And this is done in a exploratory manner.
Scripting is a main focus on our teams now ( I’m trying to manage around 10 teams at the moment directly and a lot more indirectly but that’s irrelevant ) and we are improving but as said above we have some nice challenges to overcome.
Thx for your replies
I did find my self andmy team in the same situation. What we did was to follow the test cases we had and mixed it with Story telling to add some fun to it. Similar to the feature/category arrangement @brian_seg mentioned.
We also use TestSphere cards as guides to rest from test cases, since they give interesting context as to why perform X test.
Hope this helps.
@ahopkins180 testshpere cards … Why didn’t I think about that myself …
Now you know . Best of lucks!!!
It better get automated, but in the meantime.
Maybe try to get some business users to run some, developers run some, BA run some, let testers to explore. Focus on the risks and assign each person an functional area or an user journey not test cases.
Maybe get them together have a big bash session with food as well as individual time to quietly explore
@testerawesome getting Business and Devs to run regression testing … I tried and I gave up. It is costing me too much energy …
However, I like the big bash session … good idea
I can’t believe regression testing done right is boring.
For the regression testing to exist an incremental update of the application has to have been made.
So, some exercises I do very often:
- find out what changed in between - as much as possible - business side, technical side for all available systems making the application;
- do you understand what all means? Do you need to learn more about certain aspects, sync with someone or understand an api/pieces of code? Jump on it;
- understand what was the desired scope of the changes? when are they supposed to be released? who needs them, in what context? If the desired outcome was misunderstood and mismatches the actual product behavior that seems like a regression of some sort;
- talk to the devs - they can share what they changed/worked on? the more info you can get from them and you can actually think critically about, the better.
- while you learn and experiment with new things, you should get some revelations - see bugs that were already in the system but none noticed them, see bugs reappearing, or some new symptoms of existing bugs that weren’t fixed yet, see inconsistencies in the desires of some stakeholders or users and actual product behavior.
Regression testing is, most of the time, more challenging detective work than new feature building.
That’s because you get to check in the known knowns for unknowns.
I like to focus on challenging the application, which will cover at some point the validation of the knowns as well.
we basically already do all the stuff you mentioned. The problem lies in the repetitive actions. It’s fun if you start finding defects, less fun if everything works ( at least that’s the feedback I get from my teams) )
But thx for sharing
You could try to slim down the mandatory suite of cases and then timebox half of your testing time to testing “around” the features that have changed since last release in an explorer style. But be prepared to dig up really old bugs and annoying ones too. It is risky to cut regression testing a bit short, but doing a “UX-focused” theme is another way of keeping it fresh. By labelling a regression session as UX testing, you force the testers to think about what they are seeing, which can also be a downside, but not if you prepare them a bit beforehand.
I suspect that anything of a fresh angle that you do that can be percieved as change will have the *lightbulb effect and be a positive.
*lightbulb effect : https://en.wikipedia.org/wiki/Hawthorne_effect also known as the observer effect.