When I find a bug on a product that Iām a customer of, I make sure I give as much detail as possible to help replicate the issue. Iām a tester though! Not all customers can or know how to do this.
I saw a conversation pop up recently which started with:
We have tickets reported (with a video showing the issue) by customers. We QAs are not able to replicate these problems. We tried a few times and we canāt. I know this is a very broad question, but sometimes it gets difficult when there is pressure to resolve those and it is not possible to replicate. The majority of these bugs are front end issues.
You need to get more information from the customer. What browser/system are they on, whatās the setup. What version of the product. Iāve always found in these situations Iām missing something!
In random order, based on the context it can be, among others:
change the testers - they are probably not doing a good job;
change the way the product is tested by the testers - the way that theyāre testing is having lots of holes in information gathering, for the bugs to creep through; once in production, they also canāt understand the product, risks or how it could fail.
ask for more information - it can be that the information you have is incomplete; the bug might have been triggered/started before the user thought and reported; and you only have a symptom of it.
do a live session of debugging with the user;
add loggers to the application that are sent back to the application server - dig through the logs and try to find the case, scenario, steps, problem, code causing it, etcā¦;
rethink with the other product representatives, if itās worth investigating into finding the cause; how many users have the problem? how much does it cost if not found/fixed? how much does it cost to find and fix it? how often does it appear to the same user?
guess/brainstorm - even if itās part of the above point - āchange the way the product is testedā - this can deserve a special place; many times I managed to find problems by guessing a configuration in a tool/system used by the client - based on the symptoms provided - even with just an image(no other info); the guess is usually based on going a level deeper where you havenāt been or thought of before - might require a bit of research;
go through the bug/report brainstorming out loud possible causes with someone else - sometimes your knowledge of the product business, backend systems or code, or technology, is not enough or not readily available to you. We play this game often in our team and the person thinking out loud either finds the solution themselves, or a stupid thing mentioned by someone else triggers an idea, or someone says something that could actually be linked to the problemā¦
the product has been updated and the bug was fixed; are you looking at the same version locally as the client WAS using/has reported?
Iāve been there too often. Some of the things Iāve learned to check:
What is the customerās configuration? Sometimes Iāve had to go as far as taking a copy of the customerās database/configuration/everything to be able to reproduce customer problems. This can include setting up multiple interconnected applications to match the customer environment.
Can I get detailed logs? There are times when the logs will show when something went wrong - which may be several steps before the customer sent the video.
What is the customerās operating system? Browser? - sometimes bugs will show in one browser or OS and not in another.
How exactly did the customer get to the error condition? - Iāve run into bugs that happened only if keyboard shortcuts were used to invoke a screen where clicking the button worked without issue - and vice versa. Sometimes the navigation method isnāt visible from a screen video.
Is something else masking the bug? - Iāve tested fixes for what looked like the same bug several times over, with each iteration getting more specific, until we finally narrowed down to the exact setup the customer had. The more generic versions of the bug masked the more specific versions until we fixed the generic ones.
As a general rule, if the customer can reproduce something and the testers canāt, thereās missing information. Exactly what the missing information is depends on the system and the customer.
Kate, I agree with you completely. Iāve seen several bugs related to the end userās configuration (different countries have different date formats for example), issues with IP services in other countries, and problems related to the users Mobile Device Management (MDM). These arenāt necessarily issues with the software and arenāt easily replicated or tested for, in my opinion.