When to hire a/more testers

I had a friend come by last night, heā€™s seeing some pain within his organisation around Quality, or lack thereof. He explained how the testing team is being swamped and often bypassed because their backlog of things to test is too long and he wanted to know what is a good Dev:QA ratio.

I wasnā€™t able to give him a clear number as it very much depends how testers function within the organisation and how mature the culture of Quality is. Most places iā€™ve worked tend to have one QA per 4-8 developers. The QA is embedded and responsibility for Automation is mostly with the developer using QA assistance.

What kind of ratioā€™s do you work in?
how do you determine when to hire more QA?

That is a tough question to answer, even if you know the context.

I will give an answer from my past to demonstrate. Once, I was in a small team of testers. We tested with a quite large group of programmers, and every tester tested the full stack plus. Naturally, since we had several products with different contexts, we were swamped. The management denied our requests for more testers. Quite the opposite, they took a tester away.

So we couldnā€™t have more people for testing, and we could barely keep up with demand. Our tests, while they looked good on paper, were mostly shallow. We couldnā€™t keep up with the demand while doing good testing.

So what did we do? We changed how we tested. We removed some of the formal aspects of testing which caused a lot of overhead and replaced them with more test time. We moved from the test strategy-test plan-test-execution test-report paperwork storm and replaced it with more exploratory testing, test sessions, and story telling styles of reporting. We added advocating for testability to our list of tasks. This gave us enough time to test more thoroughly, and took a lot of weight from our shoulders.

But changing your testing mindset will not always add value to your process. Sometimes, you really need that extra mind to help with testing.

5 Likes

It depends a lot on dev process in specific company. I work for a small company which has 14 developers and Iā€™m only tester there. Weā€™ll have to upscale at some point, but so far things have been working out well. Of course, I work on several projects at once and prioritize depending on sprint deadlines for specific project.

Our developers handle unit testing, integration testing and code reviews, so Iā€™m only manually checking app once it has enough functionality to be tested from end user perspective. We have two project managers that are also actively helping with testing for projects they are leading, and I donā€™t have to write test documentation so I can invest that time into more testing. Iā€™ve also started tampering with E2E automation in my spare time, but thatā€™s still not part of our regular dev process.

One of our project managers recently moved from another much bigger company and Iā€™ve talked with her about QA process there. She said testers have more responsibilities there, but they also have one dedicated tester per project.

1 Like

Like people on the thread have already said, it depends on the context. How much testing is carried out by other roles in the team? Iā€™d guess if unit tests and integration tests were carried out by developers then you could have a lower number of testers to developers ratio.

In my last role our team was 3 developers to 1 tester and that was a bit of a problem when I was solely responsible for writing automated integration checks, maintaining test environment (including writing test tools) and exploratory testing.

When I started to share the load a bit (e.g. discuss testing strategy with developers so they could write automated checks) then that ratio became more comfortable.

1 Like

heā€™s seeing some pain within his organisation around Quality, or lack thereof.

Just to be clear thatā€™s not a tester problem, thatā€™s a company problem.

He explained how the testing team is being swamped and often bypassed because their backlog of things to test is too long and he wanted to know what is a good Dev:QA ratio.

Not where Iā€™d start, nowhere near. ā€œPerson-hoursā€ is rarely the problem, and guess what happens when you throw more bodies at something that doesnā€™t work? You get 5 road workers staring at a hole drinking tea. Iā€™d look at whatā€™s in the backlog and find out why itā€™s there. Good testers should work within the boundaries they are given. Working with resources available, or ā€œlogisticsā€, is a vital part of a good test strategy. So why are they insisting on using so much time?

If thereā€™s a process making them all slave at the rock-face of pointless work, that would be a great thing to change before they spend a few grand on hiring.

  • Are they made to fill in pointless sheets or other busywork that kills the time of a skilled professional?
  • Do they have a prescriptive process of filling out test case templates or automating everything thatā€™s thrown at them or something else from the 80s that needs to piss off and die?
  • Is there a test manager, and are they awful?
  • Are the testers awful?
  • Are there giant walls between the testers and the teams they work with? Do they ever actually talk to each other? Sitting people in close proximity and hoping that information will travel via pheromones sounds like the ramblings of an insane person, but Iā€™ve seen it implemented as a business strategy.
  • Are bug reports flung over those walls over and over again like some kind of tennis match in hell?

etc. This is the role of your team managers, project managers, test managers, etc. Itā€™s what their highly trained, shiny shoed, kale-eating bodies are built for when theyā€™re not sporting fashionable shirts or hitting the squash court.

Hereā€™s some stuff one could do instead of spending 20% salary on recruitment, Ā£15k on a probationary salary, Ā£2k on national insurance, Ā£4k on pension, Ā£800 on onboarding and training, Ā£1k on office space and equipment plus holiday cover, health care, gym membership and the drinks in the free fridge, then waiting for the teams to settle back down after you disturb the workflow by throwing people into it, then repeating the process when people start leaving because your processes are still broken.

  1. Examine team processes and see whatā€™s taking so long. Check for overly formal processes, busywork and timesheets. If you make testers fill out little forms any more than absolutely necessary you might as well take their wages down to the casino instead. Burn the paperwork in the car park in a ritual, wearing board marker as face paint. Look for large feedback loops. Look for communication failures. You know whatā€™s faster than a 2-page bug report? Finding the bug mid-coding and talking face to face and it never going into a build. A company-internally-written bug report is a failure of communication and process that should be minimised.

  2. Examine the backlog and see whatā€™s in it. Why is it there? Ask your testers or test manager this question: ā€œHow does this serve your test strategy?ā€. You may also need ā€œwhat is your test strategy?ā€. If it isnā€™t serving a test strategy then itā€™s pointless - you need to find out why it got there and why nobody threw it away.

  3. Examine your dev team and see why theyā€™re moving so fast. If they are in any kind of even slightly agile environment why are the coders picking up work when thereā€™s a blockage in the pipeline? Have WIP limits and enforce them. Visualise what gets tested and do not permit anything to escape without test. Hold teams accountable when bugs escape - did they test it? Who decided not to even look at it? Then youā€™ll get your testing down pretty fast, one way or the other (or, probably deservedly, go out of business). You cannot move without bandwidth and slack. If your coders can spare a second for the proles in the testing pits perhaps they could help clear the right side of the Kanban board? Throwing pottery orders at some busy potters is not helping to make more pots.

  4. Examine your release schedule. Are you releasing often, or are you going for a more traffic-jammy approach of letting everything build up like Augustus Gloop in the chocolate tube? Do your testers have a soul-destroying 6 month ā€œregression suiteā€ filled with pointless nonsense nobodyā€™s really looked at in years? Try replacing that with a non-animal sacrifice, such as spilling the blood of a lettuce before the Gods - this is fast, and offers the same feeling of unearned comfort. For 5 testers earning Ā£25k a year at 37.5 hours a week thatā€™s over 8k a month. Spend it wisely.

  5. Hereā€™s a crazy idea I read in ā€œWild Management Ideas So Crazy Youā€™d Have To Be From The Futureā€: Ask your testers what is taking so long. What do they think would be a better, faster, less wasteful way of working? How about the rest of the team?

A business needs healthy FLOW. Unblock whateverā€™s in its way, or pay for the blockage.

Find all of those problems and fix them until your processes look like youā€™re actually achieving something, then you can see much more clearly where the issue is lack of bodies. THEN you get to think about hiring, although Iā€™d still go for a second opinion (<== this is the answer to your question of when to hire a/more testers)

Remember: When doing something stupid, more people means more stupid. When doing something clever, more people means more problems.

7 Likes

Thanks Chris,

I love your response, beautifully crafted and colourful while making very acute points. A very valuable lesson in avoiding more stupid, from what i understand of my friends organisation they may not be mature enough to take this approach but iā€™m sure it will be useful to him and we discussed many of your points although not with such wonderful colour.

Iā€™ll definitely be reaching for the marker pens next time i go to war for quality! haha

thank you so much! pure awesome all the way through.

1 Like

Thanks for the helpful replies everyone. I should add some more context which I hadnā€™t given @watchful_tester earlier.

The company in question is new to agile, new to releasing, and doesnā€™t yet even have a ā€˜we make productsā€™ mentality. These are all cultural changes we are working on addressing.

We also have 45 ā€˜creatorsā€™ (people making things that need testing in some form) and 2 ā€˜testersā€™ (and a handful of part time uni-student interns).

Our team of 2(ish) are not spending their time filling in meaningless forms, or doing busy-work. 150% of their time is spent just testing all of the features that are thrown at them, or doing regression testing to ensure that something that one of the 45 ā€˜creatorsā€™ did in the last few weeks didnā€™t break the entire product. These features are promised by higher-ups to customers.

Iā€™m all for increasing our capacity in many ways - hiring, better processes, or more likely, both - but we have to make an argument to the higher-ups about why we even need this.

My biggest argument at the moment is that we need to care more about quality.

1 Like

So your question is not when to hire more testers but how to persuade people with money to spend it on testing humans.

Iā€™d first of all show that youā€™ve done the due diligence on having a solid and efficient flow, then show the bottleneck in that flow. Unless you can show that capacity is a problem then hiring will not be the solution. If someone else can suggest a cheaper alternative then that argument has immediate appeal. It is very, very expensive to get good people and not just about the money.

Two words to loosen purse strings are ā€œriskā€ and ā€œcostā€. Explain the risks of allowing untested stories to escape because of a lack of capacity, the increased pressure causing delays in regression that incurs a person-cost. Bugs in production have a cost in terms of customer confidence, as well everything associated with fixing production code and the long feedback loops of discovery when itā€™s too late. If youā€™re going to avoid going too slowly by going too fast youā€™re going to need to even out the capacity.

Another word is ā€œscalingā€. If your company is making money and nothing seems to be going wrong youā€™re going to struggle to make the argument that things should change. If you foresee the problems you will have when you scale the current system up then you have to make that argument. If nothingā€™s wrong now and nothing will go wrong then there is no reason to spend money to fix it - and if anything needs a reason in business itā€™s monetary cost. Imagine youā€™re selling Scrooge McDuck a security system.

Whoeverā€™s going to present these arguments should write down their findings. Sometimes money people like to throw a smoke bomb to confuse their enemies, change the subject, then run, their soulless laughter echoing around the boardroom. You will identify your detractors quickly. Having a solid argument to return to can be important. Also sometimes they like to send a team of stern looking people to deflate requests like this, and having the facts to hand is marvellous boon. And weā€™re always on the lookout for marvellous boons.

If you donā€™t have a trusting relationship and some equity to spend with the human-buyer at your company be sure to send someone that does. If you donā€™t have anyone then make your argument very solid and build an image of responsible awesomeness - this is a good time to use businessy words. Someone has to be the management go-between and connect the world of making things to the people that keep all the money from making it. That takes either faith or perceived competence in that person and money people donā€™t like faith. If you have ideas about how to go about cheaply procuring new talent, at least a sketch of your intended onboarding process, what the training will look like, and other nightmare tasks, then that will help remove the counter-arguments to hiring and make people feel that they would have to do less work to agree with you.

I like people that care about quality. But the problem with arguing for caring about quality is that some uninformed people associate quality with testing, not with everyone from the CEO to the janitor like it should be, so then it becomes testā€™s problem and accusations fly while the accountant crawls under the gunfire with their armfuls of ill-gotten treasure, stopping only to scoop a stray emerald with a silver chalice. Or they donā€™t, and ask what we could do with the rest of the team instead of hiring, which is why I advise having done that legwork first, twice. Also arguing to care more about quality is indirectly accusatory because it says ā€œwe currently donā€™t care about quality enough, under the current Juntaā€ - which can fling loose from the tyre of office politics the tedious mud balls of how we DO care about quality because we have adopted a pointless standard and the test team gets free fun-sized Snickers bars and so on. Actually, arguing anything can be accusatory because it can say ā€œwhoeverā€™s job it was to balance the teams is doing a bad jobā€, so tread with utmost care and tact - you donā€™t need someone saying ā€œyou are wrong because Iā€™m not awfulā€. It depends on what your regime party members are like - best to look at the hierarchy and go to the right person with the right idea. Itā€™s not the idea that will get things done, itā€™s people, with power, that believe in it. If youā€™ve ever heard someone say ā€œI donā€™t know why they donā€™t just do Xā€¦ā€ then stood back and watched their idea take flight and triumphantly do absolutely nothing because itā€™s nobodyā€™s responsibility to care then youā€™ll understand what I mean.

1 Like