Let’s say you’re starting a new project.
A big, complex product built from scratch, with lots of complicated integrations.
You currently have zero automation
You have one dedicated QA Automation Engineer ready to start building
The goal?
Build effective automation and speed up testing (aka shipping)
Show measurable results within a few months to prove the investment is worth it
So… where would you start?
You’ve got two options. Forget the “do a bit of everything” answer, you’ve got tight deadlines and limited resources. You need a smart strategy yesterday.
Would you:
Go all in on AI R&D - research and test promising tools out there to speed up building tests from scratch, to shortcut the process with the right tool (if it exists).
OR
Invest in the classic test automation pyramid - invest time in frameworks, pipelines, unit tests, and make automation a shared team effort, even if it’s slower at first.
You can only afford to go deep into one.
What would your strategy be?
Is this a question for the project manager, the developers or perhaps a tester who has say over the overall testing strategy?
Lets assume good practice starting point based on it being a new products and you can make good choices without past mistakes, as a developer you will be automating unless something currently unknown is blocking you from doing so.
Having a dedicated automation engineer is a bonus to support them and the tester at times, in many ways already in a better position than most product teams despite the tight deadlines.
All in on R&D on a tight deadline complex project is likely high risk, depending on risk and constraints you may take a fail fast route, lets go all in for a few days but if no progress is made then go for tried and trusted approach.
I got a demo of Tester H last week and I saw the glimpse of a better AI / Agentic testing future. It looked so easy. It would be hard to say no to this and yes to more traditional automation, especially when AI tools can output code too.
I’m not sure how it compares to what else is out there, but it sure does look like an appealing way to do and manage testing.
Because of this lot has to be compromised on quality
May be you can look for good collaboration with developers so they have well covered unit test so no basic error to be caught by automation but unit test itself.
Whatever is not covered in unit test should only be automated and at UI level very basic.
Because you cannot do everything with constraints but
“do an important bit of everything”
Automation mostly helps in Regression. New development testing can take time.
Also since its building from scratch so UI automation can be parked for sometime, as its going to break with changing stuff every now and then
I would get my whole delivery team together and talk about what might be the best approach for our context. As to only being afford to go deep in one, I might see if we could persuade our management to allow us to look into both. Automation is a huge investment and it can pay off big time if done the right way for the team’s needs.
For example, back in 2016, my team needed a new framework for our UI tests. Our UI had changed and our old tool couldn’t support it. We did two rounds of comparing two frameworks, each time automating tests in both for the same features we were developing in each sprint. The first go-round, neither framework seemed right for us. The second go-round, we found a winner. Everyone on the team participated. Yes, it was costly. That team is still happily using the automation framework we chose, 9 years later. That’s a win.
Like Lisa, I would work with the development team to agretan approach. I would lol to facilitate this, and I’d ask a few questions:
What experience of Ai tooling exists within the teams and what policies etc are in place? If the team and company are new to Ai, I wouldn’t push this first, but if there is some foundation to build on I would see what my options are.
What automation, or at least programming language experience does the team have? What committment is there to invest in building and maintaining automation long term? Even if I was to be heavily involved setting things up in the first months, I wouldn’t want something that is a “Ben thing”, because I’d just be building a new bottleneck.
If I had no constraints, and little time, I’d be tempted to use a few days to see how far I can get with Ai, and be willing to throw it away if it wasn’t successful quickly.