Too many bugs, what should I do?

Hi,

I have started a new position as QA Manager in a small company 5 month ago, just me and 3 other QA working on quality.
There are many bugs in production and clients complain a lot.
We have started testing many modules and found many bugs, however it’s hard to change the trend (not all bugs are fixed and delivered quite fast).

We have also automated a lot of things, data say that the situation from a client’s point of view is now better but not that much.

What should I do in order to improve again the situation ?

Developers for now don’t do unit tests, I think that it should be important to avoid having many bugs in staging environment then fixing a lot of things again and again. A shift left would be nice, but what else ?

Thank you for your suggestions !

4 Likes

No unit tests ? You are never getting out of this hole unless you require developers to write unit tests.

Once that practice has been established, prioritize the existing bugs (High/Medium/Low) and get them fixed.

2 Likes

Firstly, congratulations on your new appointment!

My advice is; You need buy in. Buy in from those who make the decisions about what to fix. In this circumstance I’d attribute bugs with cost, and demonstrate what the company could save if it fixes bug A, bug B, bug C etc - it should be known that cost saving can be time, as well as actual money.

Depending on how open minded the people you work with are, this is easier said than done.

Outside of this, you need to find the most damaging bugs, and prioritise them. What causes the most work / money / customer annoyance. I say this because sometimes you might have bugs that annoy your customers, that are easy to fix - and these wins are good to push through even if they don’t seem like a priority, it can take pressure off and give the perception that ‘things are improving’ to the customer.

But don’t neglect those bigger issues either.

Best of luck, hope things improve!

7 Likes

Hi @jean-l ,

Thank you for sharing here and asking for advice.

Such an important topic I decided to put it out to the MoT LinkedIn Group community. You can read the replies here.

I thought this comment from Brijesh Deb was helpful:

Not an unfamiliar territory at all. Been there several times.

Too many bugs in production is a clear indicator of everyone not being on the same page. So start by asking very simple questions about product understanding and risks and you’ll spot the gaps. Does everyone understand the problem and is working towards the same goal? Look at the process gaps and you’ll start seeing gaps in the in the product too.

2 Likes

Okay here’s the story as I understand it:

  1. You are finding bugs just fine, no problem there
  2. Bugs are not being fixed before being put into production
  3. There is one or more clients that are not happy about the bugs they find

So my questions would be:
Who is complaining, about what and why?
Are the complaints coming from everyone, or just one client?
What are the nature of their complaints?
What is the situation of the complaining clients? Are they fishing for special treatment or do they have genuine complaints you confirm against your own testing?
Are their clients that are not complaining even though they use the same product in a similar way?
Are the complaints about experience problems, like UX, technical problems, like crashes or buttons not working, or needs problems, like “this software does not solve the problem we expect it to in the manner and time we think appropriate.”?

Are correct business decisions being made?
Your company , by which I mean decision-makers at your company, permits knowingly buggy software to be delivered to clients who will use it. At the same time they know that the client has a lot of complaints. They are deciding to do this. Why? Perhaps they don’t understand the needs of clients or have failed to communicate them. Perhaps they believe it’s more profitable to have clients suffer than to fix the issues. Perhaps they lack information with which to make decisions. Perhaps they promised and committed to a timeline their company cannot maintain. You may need to talk to them to figure out what it is they want and what they’re willing to give up to achieve it.

Is your testing serving your company’s clients?
Are the bugs you find the same as the ones they find? Take all clients who matter and all of their opinions that are sincere and accurate, and consider what they want, and have that guide your testing so that your findings are more aligned with your company’s clients - who are also your test clients.

Is your testing serving your company’s employees?
Does everyone communicate effectively?
Why are bugs created? Pressure to move too fast? Lack of involvement of testing early? Failure to consider problems before they’re implemented? Poor understanding of needs and requirements? Bad design?
Why are bugs not fixed? Communication problems? Laziness? Lack of slack? Lack of flow? Pressure to move too fast?

So hopefully you can take these questions and ideas to company management, clients and software teams and build up a better plan of action to solve the problem. There’s lots more to talk about but hopefully this triggers some useful thinking for your situation.

6 Likes

First things first. More testing is rarely the solution to too many bugs.

Then you need to establish how much authority do you have to change the things that could solve the problems or even get the discussions going to flag what the process problems are.

If your role is primarily testing and not bigger picture process or quality management then it can be a fools errand to try and change the things you have very limited influence over.

As a team I’d go back to basics and as an entire team discuss quality goals, the practices the whole team are doing and the put together as an entire team a holistic quality plan.

So many good practices can influence quality but its team level change and not one solution fits all.

6 Likes

It’s absolutely ok to find bugs as a tester, after all that’s our job. But a tester or a team of testers can not find all of the bugs in a given program, some are bound to slip through. The more there are, the more will slip through (and the more will be found by the testers). There are two ways to go at this problem: hire more testers since the program is obviously buggy and full of bugs that have to be found, recorded and re-tested. A much simpler route is obvious: quality measures have to be established much earlier in the process. As I understand it, there are no unit tests at all. I see them as a measure that has to be done by development (since they are “closest” to the code). I’ve experienced projects without any or at least fully functioning unit tests and usually they are a mess. So, establish a high code coverage of unit tests and have them run at regular intervals (nightlies etc). Furthermore I’d suggest coding guidelines and code reviews (if not already in place) and someone experienced taking over the role as architect, structuring the code areas and features. Setting up QA measurements at development seems to me to be the fastest and cheapest way to get some more quality into the project. Of course development has to be convinced first…

1 Like

Firstly, congratulations on your new role! It seems like you have a lot of work in front of you.

I believe your management hired you in order to fix the situation where there are a lot of bugs in production, therefore, most probably, you’ll have the support you need to institute changes you propose to improve the quality of the product.

Now, you need to get meta and try to figure out where the bugs are coming from:

  1. Are features not correctly defined?
  2. Are you shipping too many features too fast instead of investing in quality?
  3. Are features not tested properly before the release?
  4. Or are there a lot of regression issues that could have been prevented with more widely used regression testing?

As you can see from the questions, the right answer on what to do first (and you probably would need to do a lot) depends on what is the reason for those bugs.

If you work in a startup there is a good probability that you are at the exact moment where the MVP starts to become a real product and, therefore, the number of bugs explode with the number of features.
If this is the case, it is important to understand that you won’t be able to fix everything immediately, however, you can start changing things and paving the way.
And it is extremely important to get the priorities right and address the most important issues first.

I’d ask the CEO/CFO how many customers were lost because of bugs and how many deals didn’t happen because of bugs and what were these bugs. This will help you to get understanding of how company operates and what process you can institute to prevent as many important issues as possible.

Then you can propose your plan and discuss what is important with the management, and then act accordingly.

Some common notes:
One of the things we see, BTW, is that unit testing is a tool to help engineers to deliver features faster. Because, in many cases they could just run quick unit tests instead of starting the whole system and go through the steps to test the feature they are delivering. So, definitely there was some kind of misconception there.

Other than that, I’d suggest trying testRigor - your team even if they don’t have automation skills will be able to quickly build tests (like 15X faster than with Selenium/Cypress/Playwrite) and those tests are in plain English, so you can involve PMs into the review of those test cases. Disclaimer: I’m a co-founder of testRigor. If you can automate majority of the most important functionality within months then at least you free your team up from the tedious manual regression.

If the issue is related to the definition of features being purely understood you can involve BDD or SDD to make sure that specs are understood correctly.

In any case, all the best with your journey, I’m happy to elaborate more if you’d like, just ping me on LInkedIn here.

1 Like