'Basic regression will be good enough'

Rants
#1

I need to vent my thoughts here. We are doing regression testing at work. To do this, we went through a preliminary list of test cases which is draft list of items (line by line, like 'create X record). Due to the lack of time, there are no explicit test scripts so it was up to the tester to do basic regression testing.

Problem:
As we go along, I feel as though we haven’t covered all the important basic scenarios. But the test lead is happy with it as long as done a light touch on ‘basic’ scenarios.

Me being a pessimist, I said

  1. We haven’t defined what ‘basic’ means.
  2. The list does not represent all the scenarios that are basic and important (in the eyes of who?).
  3. One line / each item in the list could mean multiple scenarios (there are different ways to create/edit records)

Test Lead says;
‘exhaustive testing is impossible’.

Me:
sigh I stopped making a point there as I didn’t think the discussion would be productive. Yes of course ‘exhaustive testing is impossible’, that’s not what I mean. I wanted them to see we had not defined the meaning of ‘good enough testing’, not the other way around.

Anyways, I guess it’s not my issue in the end, the test lead will have to place their trust in how we do risk analysis in our assigned test areas (if at all, because no one reads up on things like BBST/RST concepts except me in the team).

Test lead, if you’re reading this, sorry. I don’t think you see things my way. Sometimes I feel like the bottleneck within my own testing team.

8 Likes
#2

I feel ya, I am not sure any advice I can give would help, given your constaints. But I am goign to try anyway, just in case.

In such siduations in the past, the limited amount I have been able to do is:

  • Request to continue to run testing after things are “done”, in the hope I might still find issues before customers.
  • Try and articulate the risks if failuires exist in the areas already identified, but not tests. And provide this to the test lead/product owner/project manager.
  • If you are doing a next release using the same tests, start with the ones you didn’t get to use last time. Over a longer period of time you can get better aggragate coverage.
  • Try and push the test lead/bussiness to come up with a plan to improve the siduation in the future.
  • Final stage, if all else fails, start looking for a new job. Become the test lead, and change the world (might not be helpful, frustrations exist and getting new jobs isn’t always easy. but I had to throw it out there).
4 Likes
#3

A few thoughts from me - firstly, I’d suggest that rather than talking about basic tests, instead use the phrase “critical tests” to try and get people into the mindset of considering what is the minimum set of tests that will allow sufficient confidence that the application under test is still functioning as it should. That then opens up the conversation about relative risk and importance to the end user.
Secondly, carrying out time-boxed exploratory tests in specific areas of the application may be a better way of regression testing to meet deadlines; if different testers do different areas each time you get fresh perspectives in each pass, rather than someone plodding through a set of scripted tests they could do in their sleep and may well sleep-walk past a bug!

9 Likes
#4

Thank you so much, it helps to know I’m not the only one here. When I am in the wrong headspace, I think you’re right, I need to focus on the solutions. Ideally I will address all your points except for the last one. I don’t know if I have the guts to go for a test lead position!

2 Likes
#5

Thanks for taking the time to reply to me, apparently the forum says you last posted 7 months ago! Yes, we should do cross testing with the team. We don’t really talk about exploratory testing, they are pretty much in the mindset of traditional testing (write test cases and execute them). I do agree with you that language is a precise thing, I can use the word ‘critical’ instead.

3 Likes
#6

It sounds to me like a “Risk Based approach” is needed here and as mentioned by @professorwoozle, attempt to change the terminology from Basic to critical or P1 tests.

If you can then open discussions around perceived risk areas and agree to focus testing more on those areas to give confidence, this can help dissolve barriers which it looks like you have in place…

6 Likes
#7

Yet. Give it time, if you want it you will get there. You are already identifying problems with process and thinking bigger picture, so you are on the way to lead/senior level of thinking.

If you want help getting there, just ask, as you can see the community are all keen to help!

2 Likes
#8

Have you suggested any scenarios which are not covered? Have you shown the risk if those scenarios are not covered?

PS - If you are using your real name & photo here, then I suggest that you avoid doing that. You never know if the lead is on this forum too. Watch out for questions like - Junior tester disagrees with me. What do I do? :slight_smile:

2 Likes
#9

Respectfully I disagree. This is a perfectly healthy and professional discussion. If I was a lead reading this for one of my juniors it would start a constructive conversation about the problems being faced.

2 Likes
#10

Wow, this rant blew up quickly. I feel outclassed in rantiness now. Thanks for sharing, completely agree with the need to get it out and to find a way to diplomatically defuse the conflict. When speaking to management, always offer at least one solution up. It’s all context dependant how you progress now, but at least you have identified a thing that can be improved, and managed to get good advice from @professorwoozle .

I’m also a “PUT YOUR HEAD OVER/ABOVE THE PARAPET | meaning in the Cambridge English Dictionary” type person. Don’t mind if people see that I am the same guy they see at a desk nearby, although I do wearglasses nowadays. The internet has enough temporary and superficial things going on.

3 Likes
#11

Always good to rant and share your thoughts. I have been in situations like this when people are not listening. There are some good techniques you can use - one of them is to ask a question that makes them think like, “Have we covered the important basic scenarios?” Or if you see an area that hasnt been covered ask the question “If we don’t test this area what is the risk to the customer”?

As testers we are question askers and i’d promote asking all the questions. At the end of our day we need to make risks visible.

3 Likes
#12

Thanks Melissa, it’s immensely useful to learn communication techniques. You approach avoids imposing one’s opinions on another, simply asking a question makes the other person think about it. I need to keep that in mind, as well as stay calm even when things aren’t going how I expected.

I remember watching a video where James Bach was interviewed. He gave a tip for replacing ‘Why’ at the start of a sentence, because it can come across as strong (‘Why this?’ ‘Why that?’). The solution he gave was ‘Help me to understand X…’.

5 Likes
#13

Ok. But the key word is “I” in “If I was a lead…”. It would be nice to work with people who are open minded, but unfortunately not everyone is like that.

1 Like
#14

Oh really nice tip. I’m going to use that! Help me understand…

I was thinking while running now that sometimes things go completely wrong and we have to learn from them. I do understand the frustration when you see a problem and want to fix it. Lots of ideas here. Good luck and let us know what you try / if it’s a success or not. We are always here to help on the club. :raised_hands:

1 Like
#15

Totally agree with this, and I tend to follow the strategy of not putting things online (or in general, not saying/doing things) that I wouldn’t want my grandmother to read in the paper. At that point, if an employer is going to reprimand or fire me over something I did/said, it’s probably an employer I didn’t want to work with.

As for the original post, it sounds like test lead didn’t build buy in with the OP, but they’ve done their risk analysis and come up with what they believe is right (and I totally support the “each item in the list could mean multiple scenarios” - if you have a team of strong testers, this is giving them the latitude to do a little exploratory testing around this). They’ve also likely balanced the constraints they were given (staffing, timeline, etc) with evaluating the quality of the product.

OP seems to be frustrated that he wasn’t pulled in to help come up with the test plan. Instead of focusing on this particular instance, I’d suggest thinking about how to avoid this in the future - how does one build enough professional capital to have their opinions solicited in the future? Does it require becoming a lead? Something else?

There’s also plenty of ways to work at this from the other side, by finding and reporting bugs that are on the edges of the specified plan, showing that the quality isn’t great. If the product lead or whoever is responsible for this thinks it’s fine, then that would mean that the test lead was correct in that verifying the happy path was sufficient for this (and hopefully they’ll iterate in the future if they need more). Finding important issues this way (and escalating them appropriately) can be a good way to build the afore mentioned professional capital (or just as easily, burn it down if one presses on the wrong issues - i.e. being able to accurately evaluate something’s priority and severity are super important skills).

4 Likes
#16

As part of your risk analysis, if you carefully identify the scenarios to be tested on that one line item based on the changes made to code (which you have to think through and identify), you will be alright as far as regression is concerned. No need to execute all the scenarios in regression.

But you need to test all the scenarios if you are testing them for the first time.

3 Likes
#17

This ^^ . Always have a headline item or goal for every test iteration, has made my life easier.

3 Likes
#18

As others mentioned, it’s all about the risks. What are the risks?
Why do they exist? Or if you think on what changed (code, relevance for business, etc), make tests around that.
Let’s give an analogy: imagine that you had a restaurant and the cooking staff has made an awesome cake. Some changes happened meanwhile in the staff, in the supplier you use for the ingredients, etc because you have been out for a while. Now, you came back from a large period of holidays and have an event for someone important. And you have to make sure the customer has a great experience… how would you evaluate it in few time?
We would need to take some decisions for sure. But we would need to understand what happened while you were out (i.e. the changes) and the possible risks that could arise from those. You would also have to understand if your customer/business changed meanwhile, because new risks can arise from it.
Now I need to have a cake :slight_smile:

5 Likes