Exploratory Testing: One year down, but i still have some questions!

Hey folks,

It’s been a year since I’ve started my exploratory testing journey and although fun, I still have a few things that I need a better understanding of . So I thought I’d post some of the questions that have been lingering on my mind. So here goes…

  1. What is the best way to approach a bug that is hard to replicate frequently, particularly bugs that are a high priority? In addition, how long should I spend investigating these before moving on?
  • This question in particular is one that I find quite difficult to approach. When a bug is discovered by a user that I can’t replicate, I become unsure what do, as a result I end up spending a great deal of time trying to reproduce it which becomes frustrating.
  1. What can i do to find unknown risks? Just exploring or is there more to it?
  • My assumption here is that to find unknown risks I just explore, but is there any more I can do to make my process better>
  1. How can I ensure that missing/forgotten risks, will not be forgotten on the next occasion. Any tips on methods or tools?
  • So far, I just write down any risks I find on slack (as a sort of directory to find what I’ve written) but its not cutting it.
2 Likes
  1. What is the best way to approach a bug that is hard to replicate frequently, particularly bugs that are a high priority? In addition, how long should I spend investigating these before moving on?

Do you have some kind of monitoring in place, so you can investigate the logs as well? For me personally, if I spend half of my work day (around 4h) investigating one thing it is a good sing that I need to let rest of them team know and ask for a hand, a fresh perspective can be really helpful in these situations.

Try to take into account the conditions in which the defect is reproducible- are you using the same browser as the user, same app version, is it only reproducible in production and you only have access to a lower environment? Use the process of elimination and also try to figure out the exact scenario the user has preformed to determine if it’s a legitimate bug that could potentially a lot of problems or just a very unlikely edge-case scenario.

  1. What can I do to find unknown risks? Just exploring or is there more to it?

For me getting more familiarized with the domain/business logic of the app helped in this regard. Try talking to your domain experts and asking them a lot of questions, maybe do some testing together as well, if possible.

  1. How can I ensure that missing/forgotten risks, will not be forgotten on the next occasion. Any tips on methods or tools?

Think about your app and determine which areas are the most important, what can go wrong and cause a lot of problems? On the other hand also consider which parts of your app are less critical and where more risks would be acceptable. Maybe make them more official, by talking about these things at retrospective meetings and creating investigation tasks in Jira for them.

1 Like

You asked: What is the best way to approach a bug that is hard to replicate frequently, particularly bugs that are a high priority?

If you are having difficulty replicating it, and there isn’t business pressure to get it fixed before release, park it but don’t forget it. Review the bug report from time to time, so that the next time it happens, you’ll hopefully remember to stop what you’re doing there and then and think about what you just did. Hopefully, that will expose the steps to replicate that will enable you to reproduce the bug and start running its causes to ground.

I once had a bug that I wasn’t even certain existed for three months - it fell into the category of “Did I just see it do X?”, a bug I wasn’t even certain I was seeing. But because I had the behaviour in the back of my mind, I was eventually able to understand what I was doing, demonstrate it and write the bug up in a way that enabled the devs to fix it.

  1. First I try to get as much information from the user e.g. what browser were they using etc I also like to ask when exactly it happened then the developers can help go over the logs to see if they can see something happened on our end at the same time (you may have only 1 reported issue, but other users might have been affected around the same time but not bothered to report it)
    If I don’t get this information, depending on how busy I am I may just make a note of it and park it - especially if I’ve made a few attempts but can’t replicate it (I may assume that there’s something I’m missing so don’t want to waste too much time until I get the information I asked for)

Sometimes the product owner/client/delivery lead will be ask me to investigate things further even without the extra information I’ve asked for - if that’s the case, I timebox it and then record what I have tried and what happened, then add that information to the bug to help others who may later try to debug the issue.

  1. Having conversations with people in the team, saying things like “have you thought about this?” etc. I find having conversations helps people uncover more risk, as it can help trigger people’s thinking.

  2. I tend to remember these. And then when I get a new feature, I’ll write these (additional remember) risks in my testing notes