Can changing your mindset help your team tackle irreproducible bugs?

Check out this week’s article by @w4dd325, "Heisenbugs: handling software defects you can’t reproduce", to explore the challenges of reproducing and resolving these elusive defects.

What You’ll Learn:

:bug: Understand Heisenbugs: James explains what Heisenbugs are, how they differ from other types of software defects, and the “observer effect” that makes them so difficult to reproduce.

:world_map: Navigate irreproducible defects: Discover strategies for investigating Heisenbugs, including gathering context, evaluating the potential impact, and deciding when to continue or abandon the investigation.

:brain: Adopt a new mindset: Learn how adopting a mindset that views Heisenbugs as opportunities can lead to stronger debugging practices and improved software quality.

Enhance team harmony: Explore tips for overcoming the “it works on my machine” mentality and instead promote open communication and shared ownership of defects.

After reading, we’d love to hear from you:

  • What experiences have you had with Heisenbugs or other irreproducible defects in your testing work? How did you approach them?
  • Which of the strategies outlined in the article do you think could be most helpful for you when dealing with these types of challenging bugs?
5 Likes

With the observer effect I always think of the Hawthorn effect, where people change their behaviour if they know they’re being observed. Workers at the Hawthorne Works were studied to see if better lighting would increase productivity. It did. But the effect went away when the study ended, because it wasn’t about the lighting, it was about being watched.

The observer effect in asynchronous systems is a great reminder that our tools not only shape the way we work but have an impact on the system when the tool uses the same resources as the product does. Task Manager displays how much RAM Task Manager is using.

Bach’s done some great writing on intermittent problems that carry some of the same challenges as irreproducible ones How to Investigate Intermittent Problems - Satisfice, Inc.

I found a lot of state-based irreproducible bugs in software once. They appeared quickly, then suddenly hid from me no matter what I tweaked and changed. Turns out that there’s quite a unique state in software - when you first run it. You can only run it the first time once. I had to rig a way to quickly get a fresh install to track down the problems and search for new ones in future installs. It was just a testability issue, in the end. Between that investigation and some conversations we found the mechanism that was creating the space for problems to hide, and I could test for that much easier.

The primary dilemma with “non-reproducible” defects is the frustration they cause for both testers and developers. Testers feel the dev team adopts a “work for me” approach to closing defects, while developers believe they waste time addressing ghost issues.
To alleviate this frustration, we propose the following approach:

  • Leave these defects open, label them specifically, and avoid assigning them to any milestone.
  • Assign them to experienced developers, perhaps a lead, as a side project, treating them as challenging problems to solve.

The advantages of this approach include preventing disruption to milestones and allowing experienced developers, who have a comprehensive understanding of the product, to tackle these issues effectively.

1 Like

Is the cat dead or alive?
It seems everything is quantum mecanics after all. :slightly_smiling_face:

Very interesting article James!

1 Like