Sharp addition – and I think you’re right. Shared vocabulary without a shared way to read the signals is just a common language for describing different things.
Where I’d draw the line: standardize the interpretation, not the instrumentation. The shared framework is the grammar for reading signals – what a “Red” actually means, which failures are dangerous versus cosmetic, and what a tripped signal obligates you to do about it. That part should read the same everywhere. What stays local is the instrumentation – which specific metrics a team watches and how they gather them. A frontend team and an API team can measure completely differently, as long as their “this is bad” means the same thing and triggers the same kind of response.
My gut test for where that line sits: could a leader compare two teams’ quality health without a translator? If Team A’s “Green” and Team B’s “Green” mean different things, the shared framework is missing – even when they share the words. So, the shared layer is really about legibility: quality that reads the same across the organization.
One trap I’d flag, though: “shared analytical framework” slides really easily into “mandated metrics dashboard,” and the moment you standardize the specific measures instead of the interpretation, you just bought yourself people gaming the system – a measure that becomes a target stops being a good one. So, I’d hold the shared line firmly at how we read signals and stop hard before which signals you must report. That last step is where it curdles back into the rulebook everyone quietly ignores.
How to measure is a wide enough topic that I’ve built a whole framework around it — specifically for diverse organizations. Worth a dedicated thread. I will post about it later.
Now that’s a thread I’d read twice. You’ve clearly gone deeper on this than I have – so tag me when you post it (if tagging is supported), I’d love to see how you handle the tailoring-vs-comparability tension. Good conversation.
A bit of history here. First managerial role I had, one year after examination from uni and joining the BigTelecom Company in my home country, was as ”group manager”. That, in effect, did mean that I did, on a daily basis, tell the testers what they should test or write. And if anything went wrong, I was the guy the managers did hang. That was a kinda nice role to have but not a very effective system.
I was, as most oldtimers suspicious when ”agile” came marching in at the beginning of the millennium. Noone really does perfect agile nowadays but I have to say that it works, basically. And my experience testing the works of many agile teams at various kinds of enterprises, the quality strategy forms over time in the teams. What the managers do, when things go well, is to make sure the teams are well put together, that common tools are used when appropriate - a source of common strategies - and do monitor results rater than wow’s. They do take part in demonstrations and in larger planning sessions like ”big room plannings”.
There IS a problem of making management aware of the need for quality assurance. It has, every year since I got into SW. and that has to be explained and demonstrated constantly. And it might mean investment in QA activities and indeed processes and test strategies. But I see way less of that nowadays, and do embrace the basic thinking about agile strategies also affecting QA.
Test automation is a topic I do not want to include in this discussion.