When to hire a/more testers

[watchful_tester (Daniel Barrow)]

I had a friend come by last night, he’s seeing some pain within his organisation around Quality, or lack thereof. He explained how the testing team is being swamped and often bypassed because their backlog of things to test is too long and he wanted to know what is a good Dev:QA ratio.

I wasn’t able to give him a clear number as it very much depends how testers function within the organisation and how mature the culture of Quality is. Most places i’ve worked tend to have one QA per 4-8 developers. The QA is embedded and responsibility for Automation is mostly with the developer using QA assistance.

What kind of ratio’s do you work in?
how do you determine when to hire more QA?

[brian_seg (Brian Segurson)]

That is a tough question to answer, even if you know the context.

I will give an answer from my past to demonstrate. Once, I was in a small team of testers. We tested with a quite large group of programmers, and every tester tested the full stack plus. Naturally, since we had several products with different contexts, we were swamped. The management denied our requests for more testers. Quite the opposite, they took a tester away.

So we couldn’t have more people for testing, and we could barely keep up with demand. Our tests, while they looked good on paper, were mostly shallow. We couldn’t keep up with the demand while doing good testing.

So what did we do? We changed how we tested. We removed some of the formal aspects of testing which caused a lot of overhead and replaced them with more test time. We moved from the test strategy-test plan-test-execution test-report paperwork storm and replaced it with more exploratory testing, test sessions, and story telling styles of reporting. We added advocating for testability to our list of tasks. This gave us enough time to test more thoroughly, and took a lot of weight from our shoulders.

But changing your testing mindset will not always add value to your process. Sometimes, you really need that extra mind to help with testing.

[milos (Miloš)]

It depends a lot on dev process in specific company. I work for a small company which has 14 developers and I’m only tester there. We’ll have to upscale at some point, but so far things have been working out well. Of course, I work on several projects at once and prioritize depending on sprint deadlines for specific project.

Our developers handle unit testing, integration testing and code reviews, so I’m only manually checking app once it has enough functionality to be tested from end user perspective. We have two project managers that are also actively helping with testing for projects they are leading, and I don’t have to write test documentation so I can invest that time into more testing. I’ve also started tampering with E2E automation in my spare time, but that’s still not part of our regular dev process.

One of our project managers recently moved from another much bigger company and I’ve talked with her about QA process there. She said testers have more responsibilities there, but they also have one dedicated tester per project.

[alihill]

Like people on the thread have already said, it depends on the context. How much testing is carried out by other roles in the team? I’d guess if unit tests and integration tests were carried out by developers then you could have a lower number of testers to developers ratio.

In my last role our team was 3 developers to 1 tester and that was a bit of a problem when I was solely responsible for writing automated integration checks, maintaining test environment (including writing test tools) and exploratory testing.

When I started to share the load a bit (e.g. discuss testing strategy with developers so they could write automated checks) then that ratio became more comfortable.

[kinofrost (Chris)]

he’s seeing some pain within his organisation around Quality, or lack thereof.

Just to be clear that’s not a tester problem, that’s a company problem.

He explained how the testing team is being swamped and often bypassed because their backlog of things to test is too long and he wanted to know what is a good Dev:QA ratio.

Not where I’d start, nowhere near. “Person-hours” is rarely the problem, and guess what happens when you throw more bodies at something that doesn’t work? You get 5 road workers staring at a hole drinking tea. I’d look at what’s in the backlog and find out why it’s there. Good testers should work within the boundaries they are given. Working with resources available, or “logistics”, is a vital part of a good test strategy. So why are they insisting on using so much time?

If there’s a process making them all slave at the rock-face of pointless work, that would be a great thing to change before they spend a few grand on hiring.

• Are they made to fill in pointless sheets or other busywork that kills the time of a skilled professional?
• Do they have a prescriptive process of filling out test case templates or automating everything that’s thrown at them or something else from the 80s that needs to piss off and die?
• Is there a test manager, and are they awful?
• Are the testers awful?
• Are there giant walls between the testers and the teams they work with? Do they ever actually talk to each other? Sitting people in close proximity and hoping that information will travel via pheromones sounds like the ramblings of an insane person, but I’ve seen it implemented as a business strategy.
• Are bug reports flung over those walls over and over again like some kind of tennis match in hell?

etc. This is the role of your team managers, project managers, test managers, etc. It’s what their highly trained, shiny shoed, kale-eating bodies are built for when they’re not sporting fashionable shirts or hitting the squash court.

Here’s some stuff one could do instead of spending 20% salary on recruitment, £15k on a probationary salary, £2k on national insurance, £4k on pension, £800 on onboarding and training, £1k on office space and equipment plus holiday cover, health care, gym membership and the drinks in the free fridge, then waiting for the teams to settle back down after you disturb the workflow by throwing people into it, then repeating the process when people start leaving because your processes are still broken.

1. Examine team processes and see what’s taking so long. Check for overly formal processes, busywork and timesheets. If you make testers fill out little forms any more than absolutely necessary you might as well take their wages down to the casino instead. Burn the paperwork in the car park in a ritual, wearing board marker as face paint. Look for large feedback loops. Look for communication failures. You know what’s faster than a 2-page bug report? Finding the bug mid-coding and talking face to face and it never going into a build. A company-internally-written bug report is a failure of communication and process that should be minimised.

2. Examine the backlog and see what’s in it. Why is it there? Ask your testers or test manager this question: “How does this serve your test strategy?”. You may also need “what is your test strategy?”. If it isn’t serving a test strategy then it’s pointless - you need to find out why it got there and why nobody threw it away.

3. Examine your dev team and see why they’re moving so fast. If they are in any kind of even slightly agile environment why are the coders picking up work when there’s a blockage in the pipeline? Have WIP limits and enforce them. Visualise what gets tested and do not permit anything to escape without test. Hold teams accountable when bugs escape - did they test it? Who decided not to even look at it? Then you’ll get your testing down pretty fast, one way or the other (or, probably deservedly, go out of business). You cannot move without bandwidth and slack. If your coders can spare a second for the proles in the testing pits perhaps they could help clear the right side of the Kanban board? Throwing pottery orders at some busy potters is not helping to make more pots.

4. Examine your release schedule. Are you releasing often, or are you going for a more traffic-jammy approach of letting everything build up like Augustus Gloop in the chocolate tube? Do your testers have a soul-destroying 6 month “regression suite” filled with pointless nonsense nobody’s really looked at in years? Try replacing that with a non-animal sacrifice, such as spilling the blood of a lettuce before the Gods - this is fast, and offers the same feeling of unearned comfort. For 5 testers earning £25k a year at 37.5 hours a week that’s over 8k a month. Spend it wisely.

5. Here’s a crazy idea I read in “Wild Management Ideas So Crazy You’d Have To Be From The Future”: Ask your testers what is taking so long. What do they think would be a better, faster, less wasteful way of working? How about the rest of the team?

A business needs healthy FLOW. Unblock whatever’s in its way, or pay for the blockage.

Find all of those problems and fix them until your processes look like you’re actually achieving something, then you can see much more clearly where the issue is lack of bodies. THEN you get to think about hiring, although I’d still go for a second opinion (<== this is the answer to your question of when to hire a/more testers)

Remember: When doing something stupid, more people means more stupid. When doing something clever, more people means more problems.

[watchful_tester (Daniel Barrow)]

Thanks Chris,

I love your response, beautifully crafted and colourful while making very acute points. A very valuable lesson in avoiding more stupid, from what i understand of my friends organisation they may not be mature enough to take this approach but i’m sure it will be useful to him and we discussed many of your points although not with such wonderful colour.

I’ll definitely be reaching for the marker pens next time i go to war for quality! haha

thank you so much! pure awesome all the way through.

[brookw (Brook)]

Thanks for the helpful replies everyone. I should add some more context which I hadn’t given @watchful_tester earlier.

The company in question is new to agile, new to releasing, and doesn’t yet even have a ‘we make products’ mentality. These are all cultural changes we are working on addressing.

We also have 45 ‘creators’ (people making things that need testing in some form) and 2 ‘testers’ (and a handful of part time uni-student interns).

Our team of 2(ish) are not spending their time filling in meaningless forms, or doing busy-work. 150% of their time is spent just testing all of the features that are thrown at them, or doing regression testing to ensure that something that one of the 45 ‘creators’ did in the last few weeks didn’t break the entire product. These features are promised by higher-ups to customers.

I’m all for increasing our capacity in many ways - hiring, better processes, or more likely, both - but we have to make an argument to the higher-ups about why we even need this.

My biggest argument at the moment is that we need to care more about quality.

[kinofrost (Chris)]

So your question is not when to hire more testers but how to persuade people with money to spend it on testing humans.

I’d first of all show that you’ve done the due diligence on having a solid and efficient flow, then show the bottleneck in that flow. Unless you can show that capacity is a problem then hiring will not be the solution. If someone else can suggest a cheaper alternative then that argument has immediate appeal. It is very, very expensive to get good people and not just about the money.

Two words to loosen purse strings are “risk” and “cost”. Explain the risks of allowing untested stories to escape because of a lack of capacity, the increased pressure causing delays in regression that incurs a person-cost. Bugs in production have a cost in terms of customer confidence, as well everything associated with fixing production code and the long feedback loops of discovery when it’s too late. If you’re going to avoid going too slowly by going too fast you’re going to need to even out the capacity.

Another word is “scaling”. If your company is making money and nothing seems to be going wrong you’re going to struggle to make the argument that things should change. If you foresee the problems you will have when you scale the current system up then you have to make that argument. If nothing’s wrong now and nothing will go wrong then there is no reason to spend money to fix it - and if anything needs a reason in business it’s monetary cost. Imagine you’re selling Scrooge McDuck a security system.

Whoever’s going to present these arguments should write down their findings. Sometimes money people like to throw a smoke bomb to confuse their enemies, change the subject, then run, their soulless laughter echoing around the boardroom. You will identify your detractors quickly. Having a solid argument to return to can be important. Also sometimes they like to send a team of stern looking people to deflate requests like this, and having the facts to hand is marvellous boon. And we’re always on the lookout for marvellous boons.

If you don’t have a trusting relationship and some equity to spend with the human-buyer at your company be sure to send someone that does. If you don’t have anyone then make your argument very solid and build an image of responsible awesomeness - this is a good time to use businessy words. Someone has to be the management go-between and connect the world of making things to the people that keep all the money from making it. That takes either faith or perceived competence in that person and money people don’t like faith. If you have ideas about how to go about cheaply procuring new talent, at least a sketch of your intended onboarding process, what the training will look like, and other nightmare tasks, then that will help remove the counter-arguments to hiring and make people feel that they would have to do less work to agree with you.

I like people that care about quality. But the problem with arguing for caring about quality is that some uninformed people associate quality with testing, not with everyone from the CEO to the janitor like it should be, so then it becomes test’s problem and accusations fly while the accountant crawls under the gunfire with their armfuls of ill-gotten treasure, stopping only to scoop a stray emerald with a silver chalice. Or they don’t, and ask what we could do with the rest of the team instead of hiring, which is why I advise having done that legwork first, twice. Also arguing to care more about quality is indirectly accusatory because it says “we currently don’t care about quality enough, under the current Junta” - which can fling loose from the tyre of office politics the tedious mud balls of how we DO care about quality because we have adopted a pointless standard and the test team gets free fun-sized Snickers bars and so on. Actually, arguing anything can be accusatory because it can say “whoever’s job it was to balance the teams is doing a bad job”, so tread with utmost care and tact - you don’t need someone saying “you are wrong because I’m not awful”. It depends on what your regime party members are like - best to look at the hierarchy and go to the right person with the right idea. It’s not the idea that will get things done, it’s people, with power, that believe in it. If you’ve ever heard someone say “I don’t know why they don’t just do X…” then stood back and watched their idea take flight and triumphantly do absolutely nothing because it’s nobody’s responsibility to care then you’ll understand what I mean.