I’m at a spot in my current project where I could use some feedback from others who have been in similar situations.
I’m working in a non-regulated environment, but with some very technical domain knowledge (lots of specialized math is involved, as well as physics).
We have multiple products we’re creating and updating (currently two, with more possibly coming soon).
And there are 9 dev teams, consisting of about 60 developers. We’ve been growing the teams over time, with a lot of growth this year in particular.
The test team is me (as test lead) and another tester, and we’re spread across all of the teams.
My primary responsibilities are currently:
Assist in writing test plans for features within each project.
Maintain system testing infrastructure.
Advise and guide developers on writing and maintaining tests, and how to test their work.
Executing release testing.
Assisting with Test Acceptance of features.
Creating work stories for the other tester.
Monitor code reviews for testing changes.
Right now, it feels as though I’m getting stretched thin, and depending on the time it takes to do release testing or test acceptance, other responsibilities can slip. Has this been an issue for you, and if so, what helped?
Also, if I had breathing room, I’d like to add more regular testing. Does anyone have thoughts on what would be the best type of testing to add beyond what I mentioned above?
I’m really sorry to hear about the situation you are in. Testing done by only 2 people for roughly 60 developers and such a variety of products is, to say it mildly, a lot. You are not the problem the whole system just isn’t scaled up enough. When release testing turns into too much and other duties slip, it is nothing surprising at all; just basic capacity math.
Below are a few measures that we adopted and turned out effective in a similar situation:
Ensure that the priorities are communicated clearly to top management it is better to come to an agreement on what is most important to be done during the crisis instead of trying to do everything.
Transform the QA’s mentality from “doing all the testing” to “enabling devs to test well” short checklists, templates, and reviews, can effectively bring back a lot of testing that was thought to have been done.
Do not automate testing for every case instead of concentrating on the release pain points the complete automation is not the only means to relieve the last-minute pressure.
Risk-based exploratory sessions should be introduced very little effort but huge gains.
The best investment (in my opinion) is besides more manual checks for the extra bandwidth you have, better testing earlier in the workflow acceptance criteria review, readiness checks, and improved tests authored by developers.
In brief: it is better to have the expectations shifted first than to suffer from burnout. You are already doing the work of an entire QA team so don’t undervalue that.
Document what you do, and take breaks to step back and gather feedback from the stakeholders on how effective they think you have been on the 2 current projects. You may find some teams really would prefer to own their QA than to have less than useful resource for testing. at least that’s what I would hope for as a way to scale back and give yourself time to see the operational places you can add most value as a quality evangelism function of your little 2-person team.
I’ve only worked on a project with that number of developers on a National Magistrate Court system and we had at its peak 17 Testers! So to me its pretty obvious you can’t do it all, so don’t pretend you can.
So I think @conrad.braam is onto something when he suggests you’d get the most value with quality evangelism. So I would focus on communicating risk, one of those risks may be resource but make sure you put forward proposals, not just problems.
Take a step back and look at the teams. Which teams have the better quality record? Which teams need more attention? The teams are going to have to do a lot of their own testing and you can act as the coaches to enable them to do better testing. You can then focus your expertise on those teams not doing so great, getting hands on and coaching them through it.
It does sound like doing some workshops coaching on quality for the dev teams would be worthwhile and there needs to be regular opportunities you and the devs to share learning.
I mean for ideas, with the ratio of testers to devs you have, you might want to consider cross team crowd testing time boxed alongside any automated or exploratory testing thats done. I don’t know enough about the product or the company as to whether thats feasible. But things like that are quite powerful to share the responsibility of Quality and allow teams to have an awareness of each others work.
Thanks all for the thoughts, advice, and commiseration. I’ll try to address at least some things here:
Some of this is already happening, though I should make sure to confirm priorities again.
Yeah, a lot of this is already happening, thankfully. There’s some complexity with the fact that not everyone on the team has familiarity with our testing tools (coming into the project recently mostly).
We actually pared down some tests earlier in the year (not in a way I would have liked), and while I don’t think we’re quite at the right balance of testing, we did remove some tests that were probably less worthwhile.
One of the things I’ve been trying to work on has been this, actually! The biggest issue is that I’m not enough of a domain expert to understand when certain things fail.
All good suggestions! And this is helping me feel more confident in the direction we’ve been going so far.
Good ideas here! I know we’ve had some feedback on how important testing has been.
We’ve definitely got a mix of opinions on testing across various teams, which leads to some being much more proactive in their testing and in pulling the testers in.
Thankfully, that’s been pretty clearly communicated, though I still want to push and do more most days.
I’ve put forward a few proposals, and actually one thing that was requested of me was to make this post to get more feedback from the community, and it’s working.
Good idea! The current struggle I have is being able to actually monitor them all, and a couple of teams are led by people who aren’t really bought into automation for some tests. That has made some evangelizing difficult.
I actually have some onboarding training I do to help with this! Unfortunately, we’re in a crunch right now preventing me from getting people into a workshop. I’ll keep that idea for the future though!
Good idea. Right now the issue is that we’re actually multiple companies spread across a number of time zones. It’s definitely something I could pull off with some lead time, but probably not over the holiday season here in the US.
I’m about halfway through Anne-marie Charrett’s Quality Coach book and think it would be very useful for you
Haven’t been in a situation that extreme in terms of a tester to dev ratio but maybe what I learned could still be interesting to you (I blogged about the situation I was in before: How To Test In A High Developer-To-Tester Ratio )
