Errors are rich in information

Testing Thoughts
#1

I read a blog about Learning from your bugs at the start of the week and it got me thinking.

It was nice that it was from a developers perspective for starters. Also sounds like a dev I’d love to pair with!

“Bugs help us understand the system better, and they indicate how we can improve our coding, testing and debugging techniques.”

The only example I could think of that I’ve experienced was in number entry fields on our application. “e” is technically a valid number (epsilon which is a tiny number). The first lesson we learned was that our number crunching in the back couldn’t handle it. Once we had learned this we needed to figure out a way to stop it being allowed. The first sprint after this we handled it reasonably well. Then the next sprint a different dev picked up a story with a numeric entry field and a bug was raised against it. The sprint after that it happened again with another developer.

It took us almost 4 sprints to learn from our bugs. It’s a really simple example because basically I can’t think of any others.

I’m wondering if we’re now learning from our bugs without documenting it and I don’t realise it. Or we’re heading for another fall where we spend 4 sprints learning from a bug.

Has anyone else any good examples of learning from bugs? I’d be interested if there are other ways to learn from them than this method.

#2

I’ve definitely seen this behaviour in the past, actually with similar issues - how fields behave. The best way I learnt to deal with this was to raise it during stand up, when I had everyone’s attention - or making sure I actually grab their attention. Then I’d ask the question - So we’ve had this bug around twice, do we need to standardise how the input fields behave to stop this bug coming back?? - I’d direct this to the lead dev and raise it that it keeps coming back, meaning we’re losing time and possibly missing other issues because of it. It was always done with a positive spin, so no one ever felt victimized, but it highlighted it so that no dev would want to cause it again. I’d love to hear how other people handle these situations too actually, this is a good life lesson one too I reckon!

1 Like
#3

I guess it depends how it’s documented. If you know it’s an issue could you put those notes against the story before the work starts so new devs know to look out for it? Information dissemination is always difficult as documentation is often overkill and people don’t get time to read full docs, so you have to balance giving people enough info in the right place to disseminate (though that also relies of people disseminating information, which again requires being aware of what you know that other people don’t, as well as time to do it etc etc - I have a lot of feelings about documentation and info dissemination, can you tell?)

I’m planning on writing an article about this for MoT - I did start to do an exercise where I logged things I felt I’d missed (if things came back from UAT, or a dev spotted them after me etc, or bugs that regressed/recurred over multiple sprints), or things I felt I’d not tested enough but couldn’t figure out where the gaps where in my knowledge were. Then I’d do a retrospective maybe monthly to see what I could do to make sure I was more aware of my blindspots as well as our blindspots as a team. I didn’t get time to do them regularly, but I enjoyed the heightened awareness of what I was testing and how :smiley:

2 Likes
#4

It may not exactly qualify as a good example that you’re looking for, but I’ve a fairly recent blog post that’s somewhat related to the topic - “Bugs happen – learn from them”.

I’ve also recommended to younger team mates (and still to myself) to look at missed bugs with the intent of learning from them so that hopefully we’d avoid missing them in case we get the chance to test something similar.

What the author of the blog post you shared was interesting. I wish I had been more diligent when I was younger and also kept a record of super interesting bugs that I found. :slight_smile:

1 Like
#5

Coincidentally, I just saw Jerry Weinberg’s tweet on his Leanpub book titled “Errors” https://leanpub.com/errors

1 Like