How quickly do you know if a bug youâve found is high value?
Depends entirely on context and knowledge.
Firstly, value to whom? A good tester should be looking into the values of the company and people who work within it, and the values of the people who use the product. i.e. test clients. That will help to inform whether there is any threat to the perceived quality of the product.
Have you have dismissed a bug as small of insignificant, only for it to come back bigger later on?
Have you ever found a whopper, that turned out to be insignificant?
During bug investigation I will change my mind on a bug multiple times as I find new information. Knowing how a bug can be replicated, what triggers it, what doesnât trigger it, where the vulnerability might be, all may change how often I believe itâll happen, what functionality is affected or blocked, who will suffer, and so on.
Also depends on the nature and complexity of the problem I find. It could be that the cause of the problem is not tied to any particular functionality, like typos.
Bugs are also places for other bugs to hide. If they are fixed they introduce change risk. If they block functionality they create workarounds we havenât thought of, or change the workflow in a way we might not predict, or change input methods that have different input validation or data limitations. Because bugs are a difference between what we think the product should do versus what it actually does it also creates work when other testers find it again and have to search the records for its existence (or investigate and report it again) and it may be subject to change risk in unusual ways - depending on the connection to data or functionality it may behave in ways that change the nature of the risk involved as functionality and data change or are added later.
Bugs found by users also erode the confidence in the productâs working functionality. If a company canât spell âpasswordâ correctly on the login page then how safe is my personal data? If a product behaves as if itâs faulty I will question its ability to reliably solve my problem. The user perspective is not the functional capability, logical map one of a developer. I cannot think of anything a frustrated or disappointed user cares about less than the opinion of the development team, and they are the ones who pay for the product, recommend it, review it, discuss it with people and log expensive support tickets.
All I can do as a tester is to report my findings and my evaluation with the information I have available and permit people to make a design decision about it. If I do my best and get it wrong then all I can do is learn from that and try to fold that new information into future testing as I learn what people value and the level of their expectations.
A comment from @deament about bug bash sessions encouraging engagement and fresh eyes has me thinking. I probably avoid them because I assume they will find lower value bugs that introduce noise. But I could be very wrong.
Again, lower value to whom? The advantage of a variation in perspective means that you will learn what people value. Perhaps the functionality wasnât even seen as buggy until users or support staff point out that it is. You can also find maps of concentration of where bugs are found during this process and identify places for further investigation. The value of the activity will depend on context.
Maybe we donât need the focus of a charter based testing mission
I donât think I understand the question from this point on, because I cannot imagine a more flexibly defocusable structure than testing against charters. Thereâs nothing intrinsically focused about charters. The first charters I do are usually intake and recon sessions to map out the possibilities and high-level test project issues with titles like âtake a look at this and see whatâs upâ that start the adventure towards more informed, focused missions as I begin to form an understanding of the layout, interfaces, data and functionality. Lots of people testing can be a charter, too. Itâs just to inform the mission of that session to control the sprawl of the investigation.
The costs of using a lot of people are the summation of their time and the work involved in interpreting the thoughts and artefact output of amateurs into a usable format to best learn from the results. Also herding that sort of project is not easy and takes preparation so as not to waste their valuable time and effort. If itâs available, possible and considered inexpensive for the return then Iâm all for it.
