Bug Backlogs/Mountains, how do you deal with them?

I’m a consultant so sometimes I arrive at projects which are already ongoing for years. What I often notice is that they have a huge application with loads of bugs in the backlog. (Non-Blocking bugs)

They only solve them when the customer reports them. Most of the bugs are only solved when the customer reports them and not when a QA finds them on the test environment (only during new feature development & blocking bugs)

How do you guys deal with these backlogs? (I’ve seen backlogs of 6000+ open bugs)
I sometimes am allowed to introduce like 2h-4h-1d/week time slot to focus on bug fixing, for those that “we” do kind of feel important but a lot of these bugs are not valid anymore either (we most of the time can’t afford a QA re-checking all these bug reports). So there has to be a better way to deal with bug backlogs and I’m looking for tips & tricks here.

How would you deal with joining a new project and noticing there are 6000+ open bugs?
I guess some are not blocking at all and probably will never be noticed but is it worth then having them in our backlog as open? Do you maybe have a specific status for it like ‘won’t fix-until client notices’?

Tips & Tricks are welcome :wink:

4 Likes

Your backlog is almost like digging a big whole, at some point you have added so many things that time makes irrelevant that you cannot get out that hole or you waste so much time and effort pondering about how to get out the whole.

If you are lucky and fairly early in a products lifecycle its much easier, simply do not dig that hole at all.

I had a recent project which was one of the best projects I have been on in years.

We only had two bug priorities Critical and Major. If they were at this level they were important enough to do something about either that day or at least within the current sprint.

We had other bugs that were either fixed the same day by the developer often within 10 minutes of being highlighted or they would deemed as non essential clutter.

If its not important now, its unlikely to gain importance in a couple of weeks and if we added it to the backlog it would be creating waste and distraction from more important things.

My test session notes were in the system so if suddenly it did become important we could track down those for insight, but they were not on backlog distracting the team.

This model worked really well for us, keeping bug backlog to zero or there about’s empowered the team to focus on bring value.

Should note, all developers were adding regression risk coverage with each new feature so in reality we did not get a lot of bugs due to regression.

If you are already in a fairly deep hole though, does it make sense to keep digging, perhaps landfilling it may be a more productive option even if someone is worried we may just lose some important information hiding in that hole at some point in the future.

4 Likes

Hey @kristof,

I’ve always enjoyed Augusto Evangelisti’s post on reducing defects and waste. Does this happen to be on your radar?

3 Likes

It now is :smiley:

@andrewkelly2555 Thx for your feedback & making the time to reply + Welcome back! :slight_smile:

2 Likes

@kristof : I faced same issue in my one of my project.
I bucketized the backlog as:

  1. based on Priority
  2. based on date ticket created (for example need to prioritize ticket in backlog that was raised X days / Month / Year back , where X depend how old is the project)
  3. Rather then re-checking or verifying tickets again, small meeting with old team members (dev + qa) that can comment during meeting and either close or take an appropriate action item.
  4. Remaining considered as fixed as those must be either reported again if they are of any concern :slight_smile:
2 Likes

We log all bugs, no matter how minor. Each sprint I allocate a large percentage of the time to roadmap (including bugs introduced by the current round of roadmap work) and the remainder to “other”. “Other” includes third line support, self-training, technical debt (all that stuff that we all have in software that’s over a few years old and is using third party components that could be updated, tweaks we’d like to do to areas of the code when we get time, tooling etc) plus bug fixes. Primary concern is to fix anything reported by a customer as soon as it’s reported no matter how minor it is as that improves customer perception of the company as well as the product. Secondary concern is reducing the bug list.

Just a few years ago the bug list was in the thousands (a 20 year old product base and concentration on new code rather than fixing had taken its toll). Some of the bugs on the list were even for discontinued areas of the software so no one was ever going to encounter them (some for software that had never been released) so they were closed off immediately. We now aim to pick at least a couple of bugs off the backlog each sprint (at least one per dev) and through re-testing and fixing we’ve now got the list down to under 30 for the first time in at least 10 years - 15 years ago every release went out with no known bugs although I can’t see that happening in the near future as the remaining bugs are either obscure and low risk to customers or in areas we’re not updating for a while.

My advice is actually to start with the quick wins - these don’t have to be massive amounts of work, sometimes tidying up a few cosmetic bugs that only take a few minutes to resolve will make a lot of difference to a user’s perception (no one likes typos or non-standard labels etc) and will make you feel that you are making in-roads into the problem. If you can, allocate some time each week to re-testing the backlog starting with the oldest first or areas that you know have since had a bit of a make-over - I found a number of the problems no longer existed (probably had been double-logged or changes to that area had resolved the problem) and being able to close them off really helps. I also regularly publish the bug numbers to key stakeholders; seeing the progress helps them to understand why allocating time to non-roadmap work makes a big difference to the product base.

PS an approach a previous dev manager took was to simply close the entire list and see what re-appeared - virtually everything was re-logged by the Support team whereas if the list had been left open they’d just have tagged another report to the bug and it would have risen up the priority list simply because of the number of customers reporting it. Ignoring the list doesn’t resolve the problem.

3 Likes

We’re busy dealing with this exact problem at the moment.

We prioritise bugs P1-5. P1s and P2s get sorted this sprint. P3-5 get put into a backlog that without someone really driving this, can become a backlog of 100+ issues very quickly.

We recently did a bug smash which so far has worked really well. We grouped up all the bugs in the backlog logged by other departments in the business and got them to help us close off anything that isn’t a big issue and we realistically won’t fix. With the remaining issues they wanted fixed, we got them to give us a priority so we can fix them in order that makes the biggest impact to them (instead of us prioritising).

Next we did the same with the exploratory bugs we as QAs/Devs logged and we closed off tons of P4s and P5s.

Now we are down to an achievable number of issues which we are pulling 1-2 bugs into each team per sprint and chipping away at it in priority order.

3 Likes

Ditto on some of the above - we’ve swapped in a bug smash sprint for a regular sprint and that’s helped a lot. It’s also a great way for engineers to learn about features they may not work on often and it can be a fun morale boosting exercise for the whole team. Who doesn’t love seeing bugs get closed? Note that there is substantial prep work that goes into it. The QA team grooms the backlog and tags tickets that are good candidates for a smash so that when the smash sprint starts, there’s a real dev-ready list to pluck from.

Another strategy is to have regular QA bug grooming bashes. We had a pile of old bugs sitting for a long time and once we took the time to review it we found that many of them were n/a - the product had changed a lot since the bugs were reported and either they were fixed or the feature changed so much the bugs just went away. I try to empower teams to look critically at low priority bugs - if there are a lot of bugs on a feature that is likely to be sunset, teams should feel comfortable withdrawing with a note that fixing the bug doesn’t align with the product roadmap and that’s ok.

Thought of one more thing - if you have leadership buy in, it can be good to make bug tracking and reporting a part of your culture. Eg. we have a segment during sprint demos where the QA team shares the bug trends for the last 2 weeks and we also have dashboards where individual teams can see how their own bug backlogs are trending. We have a bug SLA so having reports and visibility on bugs and bug goals are how we work to hit our SLA.

3 Likes

Thanks for sharing all this, @becky_frost. Great to read about all these options and their many practical benefits. :+1:t2:

BTW, welcome to The Club. :wave:t2:

2 Likes

@theology @chrissi @robbie.falck @becky_frost Thanks for sharing your stories & tips!

2 Likes

In my current team we use Kanban, as it’s a relatively small API-based product. There was no tester in the team until I arrived, so being a nosy tester I reported a lot of defects in the past few months. Most of them were accepted as valid after being triaged by the whole team, a few disregarded ones were related to API which were planed to be decommissioned and replaced, so there was no economical reason to fix them.

As they keep pilling up, the test summary report I’m sending at the end of each sprint, started having more and more of those red charts! So, we came to an agreement, for each sprint the developers would take a few of those bugs and move them from to-do to in-prep columns, so we tackle few of them each week. After two month of doing that the reports started to look more favourable and the upper management was very happy and encouraged us to keep doing that.

These small wins are very helpful and important, especially if you have a lot of more time consuming stories (or blocked ones) in the development pipeline to get some gratification and a sense of progress. Simon Stewart referred to this as “Dopamine Driven Development”, more details are available in this presentation, in case anyone is interested.

2 Likes

We have the strategy - and it works pretty well for us - to close bugs(or at least talk about and consider it) if they are open for a certain amount of time unblocked and without progress.

For me that makes perfect sense. Just consider a bug which is open for one year without progress. Seems like the developers don’t care about it or have more important stuff to do because otherwise they would have put a least some effort into it. Seems also like the project manager/ team leader does not care because otherwise she would shift priorities so the developers have time to fix it. Also the stakeholders of the product do not even realize the bug or see it as not important enough to force some activity on that bug.
So if no one really cares why should you keep it open?

Of course there are some scenarios where it makes sense to keep it open even though there was no progress in the last year. That’s why it is important to have a short discussion about it before closing.

Also keep in mind closing is not forever. If for some reason you later realize it was a mistake to close it - just open it again😁

2 Likes