What are Software Quality Metrics? How can you track them?
This is a very open ended question.
Before I start jumping into a detailed answer, it would be useful to know what you want to know?
Do you want to:
- Gain an academic knowledge to further your understanding or work towards a qualification (homework)
- Measure and evaluate a product that you know to have problems in order to identify useful parts of the software to improve (engineering)
- Gain an understanding of how to measure and articulate the perceived quality of a product where either you haven’t already made a judgment or you are making a comparison with a competing product (research)
In any case, I would ask who is the target audience for the metrics and is there a goal you are hoping to achieve by collecting them? This will help you choose metrics that are useful.
HI Ben . Thank you so much for your response. I am new to Software Testing. While learning about Quality I came across this term. So I wanted to know more about what exactly it is.
I’ve found 3 articles you might find useful:
Thanks Ben. I will go through the articles.
Software Quality means different things to different people. A CEO looks at a completely different set of parameters for Quality than a Test Manager does. So does the development manager, and the customer.
From your question, it sounds like you are starting off with learning what quality metrics are. I would ask you for what purpose you would like to learn them. Is it because your team/manager wants you to learn them? Or is it out of your own curiosity?
Depending on the purpose, the quality metrics will vary. As Ben said, look at your target audience and their needs. Accordingly, you can come up with your metrics towards your success factors.
Hope that helps, and do feel free to ask further questions!
There is a famous meme about there being only 2 types of programming errors. This applies here, because “Software” quality metrics only tell you things about the manufacturing of a product, but we all know that the things inside the box are not the whole product. Naming and context are always important.
By changing the first part of software quality metrics for “customer” to get customer satisfaction metrics, we get for example to a far more useful indicator of business health. To be honest, I’d rather work for a company that had very buggy software and happy customers buying product all the time, than the opposite. But I’m guessing we want to only look at the software. As @bencf1 pointed out, metrics is much more about understanding exactly the impact the thing you measure is going to have on quality, and then understanding the impact of relying on the measurement is going to end up having later on. My experience is that looking at “people over process” is much more useful, and things like: How often teams hold retrospectives; how often teams make process changes (in line with agile experiments.) Are soft indicators of health that are just as useful as counting the lines of code versus unit test cases. There are so many kinds of metric that can be objectively measured, but the ones I find the most useful have been performance metrics.
In my first job as a tester we wanted lots of metrics, so things like KLOC (thousand lines of code) and measuring this, per week the number of lines added versus lines changed is a good indicator of code churn. High churn is never a good indicator. Code coverage is still probably the best metric that does not cause fights, it actively encourages writing unit tests. Another one for us was size of the binary, on mobile platforms, big binaries produce unhappy users for multiple reasons, it also impacts runtime performance, which is why I like using performance metrics. Another useful area of metrics is TTF (time to fix) and measuring the average amount of time a high priority defect stays open for. Measuring medium priority defects is often counter productive, but being able to quickly fix a major defect can be counted by adding up all S1 P1 bugs over a 6 month window or over a release.
Coming back to the measuring of time spent brushing your teeth analogy. Total brushing time
(measuring) is a health indicator, the latter (doing) is basic hygiene. Basic Hygiene is measurable, but just like measuring how many minutes per week you spend brushing your teeth is a good thing, it’s not the only way to tell when you will need to visit a dentist.
(sorry was editing, I work that way, in fits and starts)
This is the most fantastic analogy, it perfectly describes the point was I alluding to.
Thank you very much @conrad.braam for your thorough reply. This really helped me to clear my concept. Thanks a lot again.
Thank you @testervenkat for reply . I am learning Software Testing. While exploring “Software Quality” topic I came across the term Quality Metrics & wanted to understand what exactly it is.