I work with 7 teams across 4 disciplines —Development, DevOps, Data Science, Data Engineering. When I started, “quality” meant something different to each of them (when it meant anything at all). A unit test is a different animal for a backend engineer and a data scientist; a release cycle looks nothing alike for a mobile SDK team and a web team.
I ended up building a discipline-specific maturity model rather than one universal standard, but I’m curious how others have handled this.
Have you tried a single org-wide quality standard across mixed disciplines? Did it hold, or did it either set the bar wrong for everyone or become too vague to act on?
For me, there isn’t a silver bullet. If you set up separate definitions of quality for each discipline is like giving permission to never need to understand someone elses perception. Build models with the objective of producing great outcomes and evolve them collectively, but work every day to understand each other’s perception of quality. The collaboration and collective responsibility will grow.
Really appreciate this, Gary — and I’m right there with you on the destination: collective responsibility and great outcomes, built together. That resonates a lot.
The one thing I keep turning over is whether separate definitions really mean not caring to understand each other — in my experience they can just be precision. A shared vocabulary (“what do we mean by a safe release?”) feels hugely valuable and cheap to build. A single shared standard is where I get a bit cautious, mostly because I’ve watched it either set the bar wrong for someone or drift so vague it stops being actionable.
I also wonder about diminishing returns across distant disciplines. If my Data Scientists deeply internalise a mobile SDK team’s view of quality, the empathy is genuinely good — but since they rarely touch and their failure modes barely overlap, I’m not sure how much the outcomes actually move.
So where I’ve landed (for now) is something like: collective responsibility at the seams where disciplines truly depend on each other, and specialised definitions everywhere else. Have you seen a single evolving model really hold across disciplines that don’t share much overlap? Genuinely curious how it played out.
Well if you think of things like company mission statements, there is the expectation it applies to everyone. But there is always the question from some individuals and teams, “where does this mission statement relate to what I/we do”. What they do to achieve contribute to that mission statement is different and sometimes not even that visible.
Its the same with metrics, we in engineering use the DORA metrics to apply to all of engineering. If we take deployment frequency, Product are responsible for the roadmap, developers and testers producing a quality build and Platform Engineering are responsible for the deployments. Each team will hold a part of the puzzle which is different to others, but we all have the same goal - more frequent deployments. We each need to play a part.
So I think for quality you need goals that are a high enough level to apply to everyone, but embrace the fact that when it comes to the detail, thats going to be different for each team. The collaboration comes when working together to understand the challenges around that detail.
I am not really sure I’d want to do this and I also rather dislike maturity models due to past exposure.
I have though gone on a few endeavours to align teams on both quality and testing.
The testing was a bit easier than quality as I had more ownership, I started with interviews and surveys to get others current views and then took those and went through the significant differences, myths of testing and presented what testing actually was for us, it took a number of sessions to get buy-in but worked generally. Occasionally I get a team triggering me by forgetting everything we learned.
Quality though is different and with a testing hat it drifts closer to product quality discussions, I am lucky I generally work with very smart people and a lot of it is just a cases of discussions around what makes sense and making sure every voice is heard. Usually we have many aspects with different stakeholders considered, I like the value to someone who matters aspect and as SaaS its often business value to the customer at the forefront.
The maturity model, going back 20 odd years ago I was on CMMI panels rolling process out across multiple teams. A lot of time was spent on explaining the goal was not to get levels but to improve on what we do. There was also a lot of waste and the two biggest folly’s I encountered was someone trying to tell another role how to do their job or someone acting like policing with a badge. That model I do not like though if i was looking at process side of quality I like the “help the team choose the best process for job at hand, support them to implement that process well and provide a vehicle for continuous improvement” That I have seen work, it’s more QA than quality but may be very different from your intent here.
One final thing about models - trusted and empowered is a very high level but on the way there many aspects can be misunderstood as control which can create a natural push back on.
I do not envy you this task, very valuable but I stopped enjoying that bit.
This really lands, Gary — the mission-statement analogy is a good one, and the “high-level goal, team-specific detail” split is close to where I’ve ended up too.
DORA wasn’t where my thinking started, but it met me halfway once the company decided to adopt the DORA metrics. That’s when the edges started to strain: an Automation team has no production deployment in classic DORA terms, so there’s simply nothing to measure, and a Mobile team releasing once every two sprints looks catastrophic while being perfectly healthy. The high-level goal was right; the metric just didn’t mean the same thing everywhere.
That pushed me toward an approach I’ve been calling PQI — the short version is a shift from measuring states to measuring trends, so each team keeps a definition that makes sense for them while still rolling up to one comparable value. But that (and the mission-alignment angle) really deserves its own thread rather than being buried here — I’ll write it up properly and link it back.
Thank you, Andrew — the CMMI history is exactly the cautionary tale I had in mind. It’s not so much that I feared your two “follies” (telling another role how to do their job, or policing with a badge); my real concern was different — a single unified approach won’t properly address the improvement needs that are specific to each discipline, and yet I still needed to create a shared quality vision and culture across all of them.
That’s precisely why I didn’t reach for an industry standard. I took the idea and tuned it to each discipline rather than dropping one template on everyone — and I didn’t do it alone. For each discipline I built the model together with the discipline leaders, so it’s their definition of good, not mine imposed on them.
And I’ll be honest — I did build a ladder. It’s a maturity model with levels, because I wanted an improvement map teams can actually navigate, not just a philosophy. The difference from the CMMI you describe is intent: the levels exist to show a direction of travel, co-authored with the people climbing them — not to hand anyone a certificate or a badge.
This is SPOT on! I have always made efforts to use a Rosetta Stone approach. DORA Metrics is a great example but other items in the simplest form can be helpful to. Like money.
No team or company wants to spend a bunch of unnecessary money but they didn’t know where the pipe is “leaking”. I would use Bug Cost to bring together all depts . I would grab metrics that pulled info from each one to come up with this formula.
QA and Engineering have the bug count and areas with highest bug count. Support has which customers have reported bugs in these areas. Sales and Success have what these customers purchased (i.e. modules, packages, etc) and design has features or requests were made for certainly customers.
I would take all of that and the SLA it took for the bug to close and give a Bug Cost to the company. It was an estimate but it drove home where everyone fit in the machine and also how others saw the other departments, their roles in the process. That helped to define other items and quality in particular . Having something that other can relate to them in their everyday life REALLY helps in building any culture but quality culture specifically.
This to me sounds like a few “hard” guidelines should be implemented across all teams? Like how to test requirements, how to report testing. How to write defects and how many defects would block a release.
This might be relevant. The most agile team I have ever worked with were never trained in agile, nobody mentioned it, discussed it they just were agile.
My take on this was that it was not just because it was a very smart group of people but because they had been empowered with almost complete ownership of the product, we as a team set our OKR’s, chose what features to add, prioritised these, designed our own experiments and had regular improvement sessions. There were of course some level of alignment with company goals and we needed to present our product goals regularly which were on occasion challenged but the team had a real sense of ownership, it was our product we were building and not someone else’s.
From a quality perspective this pushed the bar very high, as a team we owned and had real authority to influence it and track the results of what we were doing usually with real product data, the OKR’s that we choose were often reflective of quality.
It’s relevance here is that quality without ownership or influence will often be a simulated version of quality, gamed in a way. In a lot of cases if teams are really given ownership and a real sense they can choose and make a difference then quality will rise naturally.
There have only been a few times in my career that I have seen that level of ownership, but its stuck with me and something I often refer to when I’m helping with almost any level of change management.
Discipline-specific models over one universal standard is the right call, in my experience. A single standard almost always ends up either too strict for some teams or too vague to mean anything once it’s stretched across genuinely different types of work.
Where I’d add a bit of caution is on hard guidelines like defect severity or release-blocking thresholds applied as a blanket policy — they tend to work better as a floor at the handoff points between teams than as a rule everywhere, otherwise it turns into policing rather than a real quality culture.