When being asked to provide UAT criteria for a Go / No Go grid - what would you suggest goes on there
3 Likes
At my last job it depended on number of defects found and their severity, for instance, there should be 0 blocker anf 0 criticals, less than 5 unresolved majores, 20 known minors, etc.
2 Likes
I’d push back against strict heuristic go/no-go, especially as I don’t want to ever end up being the gatekeeper for releases.
That being said, the goal for me is always passing tests, with anyone on the team (meaning devs, product, managers, testers, etc) being able to raise a concern. If that happens, we’ll have a team discussion on what the problem is, what the risk is, and whether we’re okay putting it out as is, if we need to put it out with some special notes, or if we need to delay before deploying. At this stage, you definitely need someone who makes the final decision, but on the better teams I’ve worked with, we’ve reached enough mutual respect that we can come to an agreement without needing to have a manager make the final decision.
6 Likes
I have tried to drive less of a checklist approach to go/no-go but from a test perspective, these are usually the areas which give us confidence we are in a good place:
- No P1 or P2 open defects (unless there is a P2 workaround and fix date is viable)
- All P3 and P4 open defects reviewed and accepted by the business
- All planned Testing has completed
- Performance Testing has completed and passed
- Security Sign Off
- Outstanding Risks / Issues
2 Likes
100% what Simon said.
Releasing is not easy if you have not done the same amount of test execution and confidence , that you did in the last release. You can normally guess if a release will not blow up on the number of P2’s and P3’s in this release versus the number in the last release. If it’s much bigger, the risk is almost always bigger. No last-minute feature merge or last-minute whammies is an impossible metric to capture, but it’s often your next best indicator to the bug database trending.
1 Like
Are you specifically interested in User acceptance testing (UAT?). I’d have thought you need customer feedback for that and hopefully you were doing that all the way though not at the end. if anything that’s a reason to ship early to get feedback from customers
o some sort of pilot.
but Generally QA don’t act as gate keepers anymore since we moved on from waterfall and V-model development to agile and DevOps.
So I like to think the whole feature team(Dev/test and management) decide together when to ship. Product owners are usually the tie breakers which is especially true for user acceptance and feature completeness as they manage that relationship with customers.
QA aim to reduce risk of any defects through clear acceptance criteria, early testing, effective test automation. And provide information about the known defects and completeness of test coverage. But the whole team is responsible for quality.
That’s not to say you shouldn’t have guidelines about it or documentation to say defects in new features should normally be fixed before release. And defects found by customers within X days. But I find those are usually obvious and non-contentious. I find if there is any battle line it’s around performance and security testing particular for new products and services. Often the cost is higher when products are new and the product are in pilot or beta. So there is some haggling to be done to keep technical debt under control.
But of course the company needs to make money to pay you which means balancing the commercials and quality. Usually you will loose those fights if you pick them so the better strategy is to minimise test cost as much as possible so it’s easy for teams to do the right thing. If you can demonstrate following the test strategy leads to faster project milestones then you avoid any conflicts and have everyone champing quality and testing!
3 Likes
Preach it brother Matt!
Just don’t forget to pay testers just like you would any other role, just because the operational budget for testing per capita might be higher (nooooot!), does not mean they are lesser.
3 Likes
appreciate your comments