Which type of testers are you?

Recently I had the opportunity to do some mob testing with fellow testers, and one interesting (to me) observation came up.

For context, we have this feature and we have a set of test cases that we want to go through. Unfortunately, we are under a time constraint (story of our lives), that is even tighter than usual.

When we see something that could be a bug,

One of us would want to go explore further, while postponing the rest of the test steps. (Depth first).

One of us would want to finish the rest of the test steps first before going back to investigate the bug some more. (Breadth first)

In this context, I am breadth first, mainly due to time constraints and not wanting to lose sight of the test steps.

Which type of testers are you? Breadth first or depth first?

Looking forward to seeing your thoughts.

I don’t use test cases, but I do test to a charter, so I have a mission of what I’m supposed to be doing.

The answer is that it depends. If I can leave a problem until the end I usually will but if it’s in the way I’ll do it immediately.

If my test integrity is reduced by the problem I have to look at it straight away, so for example if I find a counting error in some functionality that I’m testing it could affect the rest of what I want to achieve. The rest of the mission, or in your example the test cases, become invalid.

If it’s unrelated, so it’s a problem in a different piece of functionality then I will make a note and ignore it so that I can keep my mind on the charter. If a problem is not interfering with functionality I will also come back to it later, so perhaps a typographical error or a rendering problem that I don’t believe impacts the work.

If it’s a design bug, like if the product works perfectly while it’s solving the wrong problem, then that’s a judgement call. If it’s a small and isolated thing I might leave it, but if it’s a major or well-connected thing I will have to grab an oracle and figure out if it’s a problem because testing something we’re going to throw away would be a waste of time.

If I don’t know what it is then I will have to investigate to establish how to treat it either way.

One issue may be that working with test cases when they are unfamiliar there’s no way to know if the validity of the test cases yet to be done will be influenced by the problem you have found. You have to investigate so that you can know what to do with the rest of the cases.

It also might be about collecting information. If I find a problem I might continue testing without directly investigating the bug because I need to understand more about the system to understand if it’s a bug or not, or what kind of bug it is. Perhaps I see something unusual which could be a symptom of a deeper functional problem, or it could be a poor UX choice, or I’m just not used to how these sorts of systems usually work and I’m surprised to learn about it. I have to recognise the emotions in the surprise and shock and confusion and unpick the truth from that.

I also go broad and deep when testing, often both within a minute of each other, because it’s a powerful technique for structured exploration, so over time I got used to how to treat the things I find as they come up as I form internal models of the systems under test.

So it heavily depends the nature of the possible problem, the impact that it could have on testing, and my oracles for understanding it.

3 Likes

I’d think about it(depth/breadth) as Chris does most of the time.

I will add though, that for me feelings and state of mind at the time might interfere as well in pushing me to the breadth or depth at various times. Am I rushed, tired, or cranky, have I argued with someone today, did I have a relaxing time ahead, do I need to prove something quickly, can I just do whatever as it doesn’t matter to anyone else how awesome my testing is, is it morning or evening, how often have I been interrupted and with what info, am I excited of a potentially huge issue I found and so on.

I’m not the best at controlling all the time the decisions that I’m making while in the moment/flow.

2 Likes