Have you ever stopped a release because of an unexpected bug, found at the last minute?

Why I ask this question:
Sometimes, that final release decision became really critical for testers to make: to ship on time or to hold back for quality. So I would love to hear stories from you, what happened when you raised the flag and stand on a product bug: how did your team respond?

I would really appreciate learning from you!

4 Likes

Yes, I’ve been in that situation.

Once during a release, everything looked fine until the final round of sanity checks when we stumbled upon a critical bug in the search filters; users could search, but the filters weren’t working at all. Imagine going live with an web app where customers can’t narrow results. That would have been a terrible first impression.

It wasn’t something we could patch later; it needed to be fixed before going live.
At that point, we had two choices: push forward and risk user trust, or hold the release and fix it. We chose to stop the release. It was stressful, but looking back, it was the right call.
Users never saw the issue, and our team learned to tighten pre-release checklists and add a few extra eyes for last-minute validation.

The key takeaway: stopping a release feels tough in the moment, but protecting quality and user experience is always worth it. Deadlines can be adjusted, but once trust is broken, it’s hard to win back.

3 Likes

It’s not the tester’s job to decide whether to launch or not, and they should not do so. In most cases there will be a product owner or project manager, and it’s their responsibility. The tester’s role is to provide them with information such as known issues, unknown factors, risks and consequences. This is only part of the information the product owner will use to make a launch decision. There will be business factors that the tester is unaware of.

Some years ago, I was the lead tester on a new e-commerce website for a very well known London department store. The website was in a disastrous state with hundreds of significant bugs including loss of data and incorrect calculation of shipping and VAT costs. In any normal situation, it would be an easy decision not to launch, and we had already missed many launch dates.

However, the project was months late, and delaying the launch to fix even a handful of the issues would mean losing substantial sales during the peak Christmas period. They had already lost a lot of sales during the summer. The chairman told the e-commerce director that he would be fired if the website did not launch by October 1st - his predecessor had already been fired after losing sales during the summer.

That was the only factor the director took into account when making the launch decision. The website launched with so many horrific bugs that it was necessary to warn the customer service team to expect a lot of complaints, which did happen. Most of the bugs were still present months later, but most of the expected Christmas revenue was achieved even if the poor quality impacted sales to some extent.

2 Likes

100% with you.

Our role as testers is being an advisor. “I’ll be you, I wouldn’t do it. But I’m not you and it’s fine.”
We are the providers of confidence. We don’t green or red light but we help people doing it to do their job with the right amount of information.

1 Like

I’ve never personally stopped a release. It should never be on one person to stop a release. But we do share information that we know is highly likely to have one outcome.

This has happened more than once but this is when the relationships that we build are vital. It lets me share what I’ve found, or been told about, in a way that will be taken as it is. I’m flagging a serious bug. Thankfully before we released.

It happens. We should then fix the issue, learn from it and move on.

1 Like

From around 2005 to 2011, I worked at two companies where it was our role as the QA team to make the final release call. We’d release every two weeks. And often we’d stop the release due to an unexpected major bug.

While many folks baulk at that approach, looking back on it now, we were probably best placed to make those decisions. We had eyes on so much of the systems and had a smooth feature and regression process locked down.

Heck yeah we felt the pressure, but the power and respect we got as a team was alluring. I’d never deny that. It was an important part of my career, and the folks in my team. No doubt we became better professionals because of it. That level of focus and pressure certainly taught us how to test and the importance of quality.

Of course, later companies and contract gigs were different. It was all about educating folks on spreading the quality and testing joy across many roles and disciplines. :smiley:

I do wonder sometimes if the next gen of testers rely too much on shared responsibility and shared ownership. :thinking:

3 Likes

Yes! The details are too specific to share on a public forum, but it was a critical bug hard to reproduce in the test environment.

I was working from home on that day, and I called the PO immediately after the discovery. They decided to postpone the release.

1 Like

Something similar to this:

2 Likes

Simon, did you have any visibility of the other factors that should be taken into account in a release decision, such as all the business factors? It’s inconceivable that quality was the only factor taken into account. Even releasing without all the planned new features isn’t a decision testers should be making, but perhaps every release candidate always included every planned feature.

And what did the product owner think about not having the final say? If I bore the ultimate responsibility for a product, I wouldn’t want someone else making the release decisions.

I was working as developer in a government project where we were renewing competitive bidding system for public sector. Near the deadline of the project we noticed one area of the new application was really broken, not just one or two bugs but multiple issues here and there and the development team didn’t feel good about pushing it to the production. We escalated the issue to the top and still got reply to go for the release. The reasoning was that they knew from the history data that nobody would use that part of the system for the next two months due to some legislation.

They were right, we deployed the new version to production in time and fixed the broken part next month before any customer used that area.

It is good to escalate because they have information which may not be available at the development function.

3 Likes

would it have been possible to disable that specific feature for the 2 months?