Bug trackers in agile, are they really necessary?

[markjnicoll (Mark Nicoll)]

Having followed a lot of conversations and read various articles, the term “bug tracker” appears in many forms, but it got me thinking, are they really necessary in an agile environment; in fact, are they necessary at all in any environment?

“What? No where to put my bugs!!” I hear you scream. No. There will still be somewhere to put your bugs, just not in an endless pit of problems where developers are too shy to enter.

Hands up how many people out there can say their bug tracker (whatever software you use) is devoid of old bugs which are now probably classed as “tech debt”? Old bugs being more than 1 month old and then some.

There are a lot of conversations about the psychology of how we all think and react to negativity, and let’s face it, bugs, even cosmetic ones, are a negative in the sense they need to be fixed by someone.

Solution to no bug tracking software

In agile especially, bugs should be added to the ticket in play and then that ticket should not be removed from WIP, but be tagged that it has bugs and await for someone to pick it up. If someone is already assigned to fix bugs as they appear, then even better, but not always the case and sometimes, it is better for the testing to be completed beforehand.

So, what happens to those bugs that you have found that are outside the scope of the ticket you are on?
Well, this is where it gets interesting. You should at least have a backlog of work. The first place, I’d suggest is for them to go here. The reasoning for this is they are in plain view and can be discussed as a part of a sprint planning session, thus reducing the chance of them getting forgotten.

Another place is it’s own column, however, I’d err on the side of no on this one as this column can end up like any bug tracking software in that it gets ignored and as such, just grows and grows in length.

Okay, so here ends the “rant/poking the hornets nest”.

Over to all you lovely peeps!

[burythehammer (Matt)]

Yes. I don’t see the benefit of having less information about the state of your product. If someone has already found the bug, you can put information about it somewhere to be retrieved at a later date.

As long as it’s properly prioritised (regular grooming is recommended), having a large backlog of bugs shouldn’t be a problem. If it turns out you need to suddenly reprioritise it to be higher, then you can do so (and you get a load of free information from when you originally logged it!)

If you have a low priority bug, then that also helps your team respond to customer issues. If someone asks about it, you have information there ready to answer it, along with workarounds. You can also add extra information about it, if someone has discovered new info.

[keiscasas (KC Casas)]

I find tracking bugs to be generally helpful in my own experience. I don’t like committing things to memory and it’s helpful when the one who finds the bug isn’t necessarily the one who’ll have to test the bug fix.

Just to share further my actual experience with agile and bug tracking… We’re using TFS for our user stories in our agile project and it also allows for us to log bugs. We’ve got a convention that if the acceptance criteria cannot be met, then we definitely have to log a bug linked to the user story. For stuff that we’re on the fence on, we talk with our devs/ux guy/po (whoever needs to be involved in the conversation) and we either add it as bug, add it as a new user story or disregard it altogether. Having the bugs linked to the user story helps us in checking whether the user story is done because if there’s an open bug then that’s a clear indication that there’s still something pending with the user story.

[johan.jonasson]

Bug trackers are generally places where information go to die, never to be heard from again. At least for the “non-critical” bugs, whatever that means.

I prefer to track bugs on the Kanban board and treat it like any other sprint backlog item. Either fix it right away or throw it away. Don’t put it in the product backlog for “later prioritization”. Unless (there’s always an unless) the bug describes a lack of something not directly related to what’s already implemented or what’s being implemented. If it doesn’t affect the next push to production then I could see an argument to save it for later.

If you’re part of an organization that likes to save bug information for “later analysis”, be honest: How often does that actually happen? What’s the opportunity cost of such an analysis? Is it worth it?

YMMV depending on the industry you’re in of course. My point is: Don’t wait to get bugs fixed. And bug trackers sometimes give a false sense of security that makes people think that we can just as easily go back and pick up where we left of somewhere down the road. We’re fooling ourselves into thinking that we’ll “fix this next sprint, when there’s more slack”. Yeah… right…

[ali1874 (Alastair)]

In our team - If a bug is introduced by a specific backlog within a sprint then it must be fixed before the backlog is marked as completed. This encourages our teams to eliminate all bugs at the time of introduction.

Of course, not all bugs are found when the main backlog is being developed. So any bugs found outwith this are added into the main backlog and prioritised accordingly.

[jrt42 (Jon)]

The issues people mention around backlogs becoming black holes is valid, but the tool isn’t the problem here. If I find a bug during a sprint but it’s not deemed of high enough priority to be fixed immediately, it’ll get chucked in the backlog.

If this backlog isn’t reviewed frequently then sure, black hole. But it’s the duty of test, dev and the product owners to ensure the backlog isn’t neglected in such a way. If it’s decided that anything that’s not worth fixing in the current sprint is never worth fixing then the bug should just be closed then and there.

If you fell asleep at the wheel of a car, you wouldn’t blame the car. So why blame the bug-tracking tool when people neglect their responsibilities in a project?

[matthewbretten]

I agree with Jon^ the issue isn’t the tool. The real question for me is - “It is worth keeping information that we no longer deem useful?”.
And that question I’d agree with Matt that having less information is worse. Sure, maybe it never gets used, but what is the cost of keeping it? Some tiny disk space somewhere?

I have had the odd occasion wheres its been useful to dig up old bugs that we never addressed, often they may include useful information regarding work-arounds or the reasons why we didn’t do anything about them the first time. I’ve had a couple of times when we’ve had some spare slack in sprints to address some minor bugs and I’ve been able to pick some out to quickly fix.

Are bug trackers a useful tool for storing and archiving bugs we aren’t working on? Absolutely, its what they were designed for! Could you use something else? Sure, but it may not be as optimal for storing and querying the information?

I tend not to use them for day-to-day work though, it usually goes something like this:
Bug reported -> Assess priority -> Triage/fix/don’t fix -> If we don’t fix now, then log it in a bug database

[alan.parkinson (Alan)]

Jon is completely right about it not being the tools fault.

For each bug on the backlog I use a best before date based on the bugs severity. Any bugs not fixed within the time period is not important and can be closed. If it was important it will be reported again or reopened.

Blog: Add a “Best before” date to bug reports

[gabenewcomb]

So many great answers here! On my current team, if a bug arises as part of a story in development, and we would feel OK releasing the story without fixing the bug first, we’ll create a separate item for the bug and put it in the backlog. As mentioned above, it’s critical to keep the backlog groomed.

I’ve also come to appreciate a heavy prioritization on fixing bugs before doing new work. I’ve worked in places where a significant amount of time was spent sifting through long lists of existing bugs trying to figure out which we could fit into our current workload, and it just felt very inefficient. So, whenever possible, fix bugs first. If it’s a bug that doesn’t need to be fixed in the next couple sprints, consider whether it even needs to be entered.

At times I’ve thought it would be a cool feature in a bug tracking system to have an expiration policy, where bugs were eventually just deleted after a set amount of time. You could really apply the same logic to stories, I suppose. (*Ah, just followed Alan’s link above, which covers this!) But good backlog grooming fixes this.

Oh, and one response to the question above about where’s the harm in keeping this information? It may be a small thing, but I find that it can create a false sense of importance around old bugs if they’re being stored, curated, counted, whatever, for months and years at a time. It also adds more cruft to search through (say you want to look up old stories and bugs on feature X…do you want to see a slew of unimportant, unfixed bugs for it?) I suppose it’s an obvious point, but managing bugs isn’t delivering value to the customer.

[robertday (Robert)]

Agreed, Matthew. Whether to fix non-critical bugs is always going to be ultimately based on a business decision of some sort that will sometimes run counter to the pure thoughts of devs and testers. I’ve always worked on the basis that all bugs are recorded; what happens to them next is sometimes going to be a business decision, and sometimes not.

If a client or user later identifies a bug, then there are good reasons to be able to come out with the sort of explanation you describe. And it may be that the user has encountered the bug in a different context that changes its criticality; in which case, it’s necessary to ensure that the full bug story is recorded.

[adrianw (Adrian)]

I’d suggest it’s essential to have bugs and stories tracked in the same tool. There should be one source of information about the state of the product.

Something we do to keep everything contained and minimise the amount of bugs raised is to write acceptance criteria in our stories at the level that allow us to reject the story if a related bug is found. If the story has a bug, then the story goes back to the devs asap.

We will still raise bugs, for example if we notice a regression or a bug that initially slipped through that was part of an already accepted story. Another key thing we do is to adopt a ‘broken windows’ approach to bugs. That is, they get fixed as a priority so they don’t hang around getting stale or growing into a big pile of guilt and failure.

We try to apply these principles consistently and it seems to work pretty well. One place where we will make an exception is for legacy code and products. If during some exploratory testing we find a bug that’s been live for the last five years, then we will log it in the backlog so it is recorded and can be discussed and searched for later, but it may not be prioritised.