Background: When I joined I was the only QA on my team, we got another and now have 3… I had a conversation today with the QA who tests one of our web consoles. She works alone now… and it got me thinking.
I work mostly on native, not web, but for this question I want to know how many testers (or developers who write tests half of the time), of any kind at all, since we are very inclusive type people here, does it take to test 1 web page. Or to phrase it better, how many pages can one tester completely own? I’m talking pages that show dynamic data, pages with input boxes/forms and any web pages that you normally would have in a page-object-model testing methodology.
I’ll go first, because I think the number of “functional and live data” web pages is probably somewhere between 15 and 20. What say you?
@conrad.connected - I think it depends how you plan for capacity/how you measure the workload for the testing as well as how the project is managed.
I’ve done this by various of ways as follows:
1.) One project I was working directly with app development team, testing the features (user stories) of the app as they were developed. It was Waterfall project, there was more developers than QA, so I determined the amount of QA needed by the workload of the developers. I figured out the ratio of QA per 3 developers was about right for test coverage needed per story & amount of bugs found for both new feature & regression testing.
2.) In a more agile project, we had a QA per feature team/pod. They would ask for support from other QAs when needed. They determined this by T-sizing the user stories and amount of stories coming, working with PMs to determine amount of testing needed by when. We worked in 2 week sprints. To make this work, we had a QA Org for different roles, so we had senior QA (Engineer) act as QA owner always in that feature team than we had more young QA that would support when needed but also had alot of tasks across project like regression testing and bug retesting.
I realize that isn’t direct answer but may give some ideas about how go about thinking of determine QA resources you need.
Agree. I think the planning and sizing of tester to dev ratio needs some up front thought, too many teams think that 1:5 is right and on the other hand I see one overwhelmed team. I do like the idea of having T shirt sizes, but I also recognise that sizes do drift a little over time. It also depends a lot on seniority/experience. And I think I’m seeing a team with fewer people struggle but mainly due to experience, not due to workload. Churn is also not helping sometimes.
I think the complexity of what constitutes a page can vary so widely that there’s probably no real way to answer a question like this without additional context. There are some pages on the internet that probably have entire dedicated testing teams or at least a team’s worth of testing resources available - e.g. the facebook news feed or the gmail inbox.
I did forsee this question I hope. A news feed is actually multiple components - if I was using page object model for the Facebook home page, it’s probably at least 5 page-objects right there for example, there are adverts, live status counters and the feed to name but a few functional areas that will break out into their own object. If we look at the breakdown that way does it help?
I think for me, this entire train of thought has made me very aware, that my team are luck to be well resourced. Teams that are not are as lucky are almost guaranteed to struggle when summer holidays come along.
By the way, standardization in pages and processes helps get this number way up. I couldn’t do half of it if I had to test differently in every interaction.
I’m guessing that as long as people stay at an organisation, the number of apps that live in long-term memory only ever increases!
On a side note, how do people tie up their testing efforts into longer term documentation that might also help with knowledge transfer to new staff, as well as useful aide memoires?
Very interesting question / take on the ratio problem. I have mostly looked at it from how many changes can one tester manage, which we can translate into how many developers can testers manage. When instead looking at it from pages, at a previous job we had one team that managed the report generation page and they were three devs and one tester (not unit tests to speak of) for one page. Another team was four devs and one testers for 3 pages which had the search functionality. And my recollection was that no team had more than 3 pages.
But here is why I think this question is interesting. Because I recently talked to a person at a different company in the same field as ours and I asked. According to you what is the shortest amount of time given a team of a specific size that you can “make a page”, wind in your back, sun is shining and all that. And they answered a time which is longer than what we currently do. So having something where we can at least revisit, our current position is 3 pages per tester, but others can do 25. What do they do differently and should we do too? That is a very good question to ask.
I only came around to this idea after I got the first few responses. I just have to admit. I am a huge fan of documenting in ways that allow other people to onboard and ramp up quickly to my level. It’s dangerous however to expect everyone to be able to work as fast or efficiently as you might I know. BUT. If you don’t spend time documenting what you do know, you might also not get that grip on what it is that you know and needs transferring to others.
I’m lately warry of documenting things too much, have seen wikis that read more like a personal blog, or that are like Egyptian (I have nothing against Egypt) archaeology. And have no bearing on what the engineer is trying to troubleshoot after a long time. In a word, “stale”. But I think tactical transfer of knowledge and continuously looking to do things like swap people between teams to allow these speed or workload comparisons to turn into actionable insights. Things like standardising not just the way we test pages on a website but standardising how we implement them too. Glad this idea got us thinking.
I did forsee this question I hope. A news feed is actually multiple components - if I was using page object model for the Facebook home page, it’s probably at least 5 page-objects right there for example, there are adverts, live status counters and the feed to name but a few functional areas that will break out into their own object. If we look at the breakdown that way does it help?
I’m not sure it does. The size of a page or page component is fuzzy and the amount of work it is to test those components is also fuzzy.
Even if you could draw a clear line between pages I’m still not sure it would help determine how many resources are needed to test them since other variables which are tough to measure like code complexity, tech debt, “bug tolerance”, etc. could vary so much.
I’m sure there is a relationship between number of pages and the amount of work, but I don’t think the formula is simple and linear.