I need to vent my thoughts here. We are doing regression testing at work. To do this, we went through a preliminary list of test cases which is draft list of items (line by line, like 'create X record). Due to the lack of time, there are no explicit test scripts so it was up to the tester to do basic regression testing.
Problem:
As we go along, I feel as though we havenāt covered all the important basic scenarios. But the test lead is happy with it as long as done a light touch on ābasicā scenarios.
Me being a pessimist, I said
We havenāt defined what ābasicā means.
The list does not represent all the scenarios that are basic and important (in the eyes of who?).
One line / each item in the list could mean multiple scenarios (there are different ways to create/edit records)
Test Lead says;
āexhaustive testing is impossibleā.
Me: sigh I stopped making a point there as I didnāt think the discussion would be productive. Yes of course āexhaustive testing is impossibleā, thatās not what I mean. I wanted them to see we had not defined the meaning of āgood enough testingā, not the other way around.
Anyways, I guess itās not my issue in the end, the test lead will have to place their trust in how we do risk analysis in our assigned test areas (if at all, because no one reads up on things like BBST/RST concepts except me in the team).
Test lead, if youāre reading this, sorry. I donāt think you see things my way. Sometimes I feel like the bottleneck within my own testing team.
I feel ya, I am not sure any advice I can give would help, given your constaints. But I am goign to try anyway, just in case.
In such siduations in the past, the limited amount I have been able to do is:
Request to continue to run testing after things are ādoneā, in the hope I might still find issues before customers.
Try and articulate the risks if failuires exist in the areas already identified, but not tests. And provide this to the test lead/product owner/project manager.
If you are doing a next release using the same tests, start with the ones you didnāt get to use last time. Over a longer period of time you can get better aggragate coverage.
Try and push the test lead/bussiness to come up with a plan to improve the siduation in the future.
Final stage, if all else fails, start looking for a new job. Become the test lead, and change the world (might not be helpful, frustrations exist and getting new jobs isnāt always easy. but I had to throw it out there).
A few thoughts from me - firstly, Iād suggest that rather than talking about basic tests, instead use the phrase ācritical testsā to try and get people into the mindset of considering what is the minimum set of tests that will allow sufficient confidence that the application under test is still functioning as it should. That then opens up the conversation about relative risk and importance to the end user.
Secondly, carrying out time-boxed exploratory tests in specific areas of the application may be a better way of regression testing to meet deadlines; if different testers do different areas each time you get fresh perspectives in each pass, rather than someone plodding through a set of scripted tests they could do in their sleep and may well sleep-walk past a bug!
Thank you so much, it helps to know Iām not the only one here. When I am in the wrong headspace, I think youāre right, I need to focus on the solutions. Ideally I will address all your points except for the last one. I donāt know if I have the guts to go for a test lead position!
Thanks for taking the time to reply to me, apparently the forum says you last posted 7 months ago! Yes, we should do cross testing with the team. We donāt really talk about exploratory testing, they are pretty much in the mindset of traditional testing (write test cases and execute them). I do agree with you that language is a precise thing, I can use the word ācriticalā instead.
It sounds to me like a āRisk Based approachā is needed here and as mentioned by @professorwoozle, attempt to change the terminology from Basic to critical or P1 tests.
If you can then open discussions around perceived risk areas and agree to focus testing more on those areas to give confidence, this can help dissolve barriers which it looks like you have in placeā¦
Yet. Give it time, if you want it you will get there. You are already identifying problems with process and thinking bigger picture, so you are on the way to lead/senior level of thinking.
If you want help getting there, just ask, as you can see the community are all keen to help!
Have you suggested any scenarios which are not covered? Have you shown the risk if those scenarios are not covered?
PS - If you are using your real name & photo here, then I suggest that you avoid doing that. You never know if the lead is on this forum too. Watch out for questions like - Junior tester disagrees with me. What do I do?
Respectfully I disagree. This is a perfectly healthy and professional discussion. If I was a lead reading this for one of my juniors it would start a constructive conversation about the problems being faced.
Wow, this rant blew up quickly. I feel outclassed in rantiness now. Thanks for sharing, completely agree with the need to get it out and to find a way to diplomatically defuse the conflict. When speaking to management, always offer at least one solution up. Itās all context dependant how you progress now, but at least you have identified a thing that can be improved, and managed to get good advice from @professorwoozle .
Always good to rant and share your thoughts. I have been in situations like this when people are not listening. There are some good techniques you can use - one of them is to ask a question that makes them think like, āHave we covered the important basic scenarios?ā Or if you see an area that hasnt been covered ask the question āIf we donāt test this area what is the risk to the customerā?
As testers we are question askers and iād promote asking all the questions. At the end of our day we need to make risks visible.
Thanks Melissa, itās immensely useful to learn communication techniques. You approach avoids imposing oneās opinions on another, simply asking a question makes the other person think about it. I need to keep that in mind, as well as stay calm even when things arenāt going how I expected.
I remember watching a video where James Bach was interviewed. He gave a tip for replacing āWhyā at the start of a sentence, because it can come across as strong (āWhy this?ā āWhy that?ā). The solution he gave was āHelp me to understand Xā¦ā.
Ok. But the key word is āIā in āIf I was a leadā¦ā. It would be nice to work with people who are open minded, but unfortunately not everyone is like that.
Oh really nice tip. Iām going to use that! Help me understandā¦
I was thinking while running now that sometimes things go completely wrong and we have to learn from them. I do understand the frustration when you see a problem and want to fix it. Lots of ideas here. Good luck and let us know what you try / if itās a success or not. We are always here to help on the club.
Totally agree with this, and I tend to follow the strategy of not putting things online (or in general, not saying/doing things) that I wouldnāt want my grandmother to read in the paper. At that point, if an employer is going to reprimand or fire me over something I did/said, itās probably an employer I didnāt want to work with.
As for the original post, it sounds like test lead didnāt build buy in with the OP, but theyāve done their risk analysis and come up with what they believe is right (and I totally support the āeach item in the list could mean multiple scenariosā - if you have a team of strong testers, this is giving them the latitude to do a little exploratory testing around this). Theyāve also likely balanced the constraints they were given (staffing, timeline, etc) with evaluating the quality of the product.
OP seems to be frustrated that he wasnāt pulled in to help come up with the test plan. Instead of focusing on this particular instance, Iād suggest thinking about how to avoid this in the future - how does one build enough professional capital to have their opinions solicited in the future? Does it require becoming a lead? Something else?
Thereās also plenty of ways to work at this from the other side, by finding and reporting bugs that are on the edges of the specified plan, showing that the quality isnāt great. If the product lead or whoever is responsible for this thinks itās fine, then that would mean that the test lead was correct in that verifying the happy path was sufficient for this (and hopefully theyāll iterate in the future if they need more). Finding important issues this way (and escalating them appropriately) can be a good way to build the afore mentioned professional capital (or just as easily, burn it down if one presses on the wrong issues - i.e. being able to accurately evaluate somethingās priority and severity are super important skills).
As part of your risk analysis, if you carefully identify the scenarios to be tested on that one line item based on the changes made to code (which you have to think through and identify), you will be alright as far as regression is concerned. No need to execute all the scenarios in regression.
But you need to test all the scenarios if you are testing them for the first time.
As others mentioned, itās all about the risks. What are the risks?
Why do they exist? Or if you think on what changed (code, relevance for business, etc), make tests around that.
Letās give an analogy: imagine that you had a restaurant and the cooking staff has made an awesome cake. Some changes happened meanwhile in the staff, in the supplier you use for the ingredients, etc because you have been out for a while. Now, you came back from a large period of holidays and have an event for someone important. And you have to make sure the customer has a great experienceā¦ how would you evaluate it in few time?
We would need to take some decisions for sure. But we would need to understand what happened while you were out (i.e. the changes) and the possible risks that could arise from those. You would also have to understand if your customer/business changed meanwhile, because new risks can arise from it.
Now I need to have a cake
