From our conversation in this Week (This Week in Testing (30 May 2025) - crowdcast) and as a preparation for my LDX3 Talk ( The million dollar bug: Quality leadership lessons from costly failures - LDX3 London 2025 Agenda - What's happening?) in June:
What do you want engineers (head of engineering) and management (e.g. CTO) to know about Quality? What is the relationship between speed and quality?
2 Likes
One of the relationships I see between speed and quality is that balanced speed always brings good quality however, if we are rushing, it might be highly possible that we are compromising quality at any point during the testing.
Delivering features at speed sounds fascinating, but it also comes with a risk to quality, which eventually end user will face while using the application.
And hence, testing strategies become a crucial component in such a place where we have to maintain speed as well as quality.
Engineers and management should be aware of good testing strategies that encourage quality in the product as well as in the team.
Along with that test planning is also important.
3 Likes
some Feedback from the chat:
@al8xr : One thing that we need to promote across the teams is the idea that QA engineers are like doctors that only highlight the problems. Itâs up to the management to make a decision whether to fix bugs (illness) or âlive with itâ.
@ghawkes : For my CTO, quality isnât automating every testâŠnow matter how many times they try to tell me. The message is getting through gradually
I think the overall message for CTOâs is that QEâs donât hold a big bucket to catch 100% of the quality problems. Everyone involved holds a different bucket at a different area of the pipe to catch different problems.
4 Likes
Here are some thoughts off the top of my head, not in any particular order:
- To have meaningful discussions, stakeholders need to agree on what âqualityâ means to them. You can be sure there is currently no shared understanding, and if there is itâs not the same as yours (or as yours ought to be). They probably interpret quality as meaning the number of known bugs or something similar.
I like the âQuality is value to some person that mattersâ definition from Kaner, Bach and others. This is broad enough to include security, performance, load, stress, usability, accessibility, compatibility, interoperability, maintainability (e.g. code comments, limits to nested functions, variable naming, indenting etc.), resilience, benchmarking against competitors and many other factors.
But some stakeholders may measure quality in terms of profit (often the organisationâs ultimate goal), subscriber numbers, complaints, net approval rating etc.
- There is no such thing as âcompleteâ testing. For any non-trivial application, you can do an infinite number of tests. Other terms such as âexhaustive testingâ are equally meaningless.
- There will always be tests you donât have time to do.
- Not every test is worth doing.
- Most bugs will not be able to be found by testing against the documented requirements. To achieve good quality, exploration and investigation must be part, perhaps most, of your testing.
- Given an infinite task, the testerâs responsibility is to make best use of the available time. Verifying the documented requirements in the order they are written is not likely to be the best use of the testerâs time.
- The inverse relationship between speed and quality is obvious, assuming all other factors are fixed. To increase both, you need to vary those other factors, such as employing better people, omitting low-value features and eliminating waste such as unnecessary documentation.
- Quality cannot be reduced to numbers. There are no valid software testing metrics.
- The amount of testing you do is always a balance between risk and cost. There is no ârightâ amount of testing.
- Anyone can do bad testing, but good testing is really difficult and most people canât do it. Get the best (and probably the most expensive) testers you can. Anyone can do bad development or project management, but you wouldnât want them on your project, so why do you think testers are any different?
- You can release the software whenever you want, even if no testing has been done or known bugs have not been fixed. The risk will be higher, but it might still be the right decision. Social media platforms are an example - releasing new features rapidly is far more important than quality.
I was working for one of the worldâs best known department stores when it became very clear the new e-commerce website would not be ready for the Christmas period. The chairman informed the E-commerce Director he would be fired if the website didnât launch on 01 October, so it did. It was horrendously buggy, including VAT and shipping cost errors, and there were loads of complaints, but the expected sales revenue was achieved. It was the right decision in a bad situation.
- You will launch with bugs. Some you may know about, the rest you wonât.
- TDD aside, itâs not usually worth automating checks until software is stable. Donât waste time automating the testing of features in the same sprint they were created, because you will need to keep fixing them and may even need to remove them. This will cause howls of rage from agile zealots who insist everything must be done within a sprint, which is stupid dogma.
- Everything is context-dependent. A practice that improves quality in one context may be a really bad idea in other contexts. For instance, Facebook used to boast that they didnât do testing (who would have guessed?), but what they didnât say is that time to market trumps everything else for them.
For further ideas, I recommend reading Lessons Learned in Software Testing by Pettichord, Kaner and Bach. Everyone ought to read it because itâs valuable beyond the context of this thread.
As an aside, the list of factors in the first list item above illustrates why I am opposed to the current fad for testers calling themselves âquality engineersâ. There are many aspects of quality that testers are entirely unaware of or have minimal knowledge of. Many aspects are beyond the testerâs responsibility or influence. Testers also lack credibility with regard to some aspects - can you imagine telling the development manager that indenting should be done using tabs, not spaces (or vice versa) and definitely not a mixture? Or that you donât like the way developers name variables?
2 Likes
We can achieve quality if managers and engineers understand that quality is a result of a process. People have been using a Plan, Do, Study, Act Cycle to improve quality for decades. The use of this cycle will help managers and engineers to achieve quality.
There are many definitions of quality. I have found it useful to have an understanding of how Deming, Shewhart, and Taguchi understood quality.
This is a talk I gave for MoT on the work of Deming, which includes his approach to quality and the Plan, Do, Study, Act Cycle: How does Demingâs work on quality help us?
1 Like