how to find out: we could ask colleagues from customer service/2nd level support.
30 Days of Testability Day 4 - Do you know what your top three customer 5 impacting issues are? How could you find out?
wondering, what this question is really about.
“Do you know what your top three customer 5 impacting issues could be? How could you find out?”
In this case it is not about knowing issues but think about possible issues.
Then my answer would be:
- check the intended use of the SUT - if the related functionality doesn’t work or doesn’t work properly, it must be a problem for users
- execute a risk assessment to identify risks which may occur if users use the application (tester’s SUT) and it does not work as required.
If your solutions are facing the general public, consider social media (FB, Twitter, etc) and “trust/recommendation” sites (TrustPilot,…).
Try it anyways, even if your solutions are a niche - people might be chatting about it.
The most immediate source of information on issues impacting customers could be the issues of top proirity in the backlog. The prioritised backlog is often the product of analysis performed by the product owner/business analyst/UX analyst.
If looking for information from outside the project team, valuable areas could be: customer service incident management system, analytics or user survey results.
Depending on the definition of ‘impacting issues’, system logs could also provide some insight into issues encountered by users.
Depending on the type of project, this may change on a frequent basis. I know what our top 3 customer impacting issues are today, at this present moment (search-ability, sort-ability, and complete/cleansed/unified product data). However, if some functionality breaks later today that is more important than the current 3 (inability to checkout, for example), the list of top 3 issues will need to change to reflect the new issue.
In my case, the customer-facing issues are most frequently reported by our customer support team. At other times, I hear from customers directly - or peers who have heard from customers directly. There could be occasion when the developers find an issue and report it. Or, I may stumble across an issue during my tests.
In my case, I could query our issue tracking system for the product areas or features with the highest number of reported bugs. More to the point, I could pick the ones with the most number of customer-created tickets linked to them (i.e bugs that had the most number of clients calling or emailing to report them)
This is a leading indicator of how close you are to those who use your application, the closer you are, your testing is likely to be more meaningful. Empathy with customers and support teams for your application will help you determine what matters to them and test accordingly. Better testability (in this case relationships) helps target risk.
On the flipside, you might have lots of silent problems which customers simply ignore or live with, gradually eroding their confidence in your application. If your system has an analytics app attached (like New Relic for example) what errors occur most often?
Use both angles to improve your testability!
From the Twitter community:
This is also the case for me, we also add in checking with our support team as well to see if there are items that customers have requested or problems that have been reported to them often but not added to our ticketing system.