How to replicate customer bugs

When I find a bug on a product that I’m a customer of, I make sure I give as much detail as possible to help replicate the issue. I’m a tester though! Not all customers can or know how to do this.

I saw a conversation pop up recently which started with:

We have tickets reported (with a video showing the issue) by customers. We QAs are not able to replicate these problems. We tried a few times and we can’t. I know this is a very broad question, but sometimes it gets difficult when there is pressure to resolve those and it is not possible to replicate. The majority of these bugs are front end issues.

What advice would you give to this person?

1 Like

You need to get more information from the customer. What browser/system are they on, what’s the setup. What version of the product. I’ve always found in these situations I’m missing something!

2 Likes

In random order, based on the context it can be, among others:

  • change the testers - they are probably not doing a good job;
  • change the way the product is tested by the testers - the way that they’re testing is having lots of holes in information gathering, for the bugs to creep through; once in production, they also can’t understand the product, risks or how it could fail.
  • ask for more information - it can be that the information you have is incomplete; the bug might have been triggered/started before the user thought and reported; and you only have a symptom of it.
  • do a live session of debugging with the user;
  • add loggers to the application that are sent back to the application server - dig through the logs and try to find the case, scenario, steps, problem, code causing it, etc…;
  • rethink with the other product representatives, if it’s worth investigating into finding the cause; how many users have the problem? how much does it cost if not found/fixed? how much does it cost to find and fix it? how often does it appear to the same user?
  • guess/brainstorm - even if it’s part of the above point - ‘change the way the product is tested’ - this can deserve a special place; many times I managed to find problems by guessing a configuration in a tool/system used by the client - based on the symptoms provided - even with just an image(no other info); the guess is usually based on going a level deeper where you haven’t been or thought of before - might require a bit of research;
  • go through the bug/report brainstorming out loud possible causes with someone else - sometimes your knowledge of the product business, backend systems or code, or technology, is not enough or not readily available to you. We play this game often in our team and the person thinking out loud either finds the solution themselves, or a stupid thing mentioned by someone else triggers an idea, or someone says something that could actually be linked to the problem…
  • the product has been updated and the bug was fixed; are you looking at the same version locally as the client WAS using/has reported?

I’ve been there too often. Some of the things I’ve learned to check:

  • What is the customer’s configuration? Sometimes I’ve had to go as far as taking a copy of the customer’s database/configuration/everything to be able to reproduce customer problems. This can include setting up multiple interconnected applications to match the customer environment.
  • Can I get detailed logs? There are times when the logs will show when something went wrong - which may be several steps before the customer sent the video.
  • What is the customer’s operating system? Browser? - sometimes bugs will show in one browser or OS and not in another.
  • How exactly did the customer get to the error condition? - I’ve run into bugs that happened only if keyboard shortcuts were used to invoke a screen where clicking the button worked without issue - and vice versa. Sometimes the navigation method isn’t visible from a screen video.
  • Is something else masking the bug? - I’ve tested fixes for what looked like the same bug several times over, with each iteration getting more specific, until we finally narrowed down to the exact setup the customer had. The more generic versions of the bug masked the more specific versions until we fixed the generic ones.

As a general rule, if the customer can reproduce something and the testers can’t, there’s missing information. Exactly what the missing information is depends on the system and the customer.