I have been - and to some extent still am - in the same position as you. 6 months ago our CTO decided the same, lets go with Agentic AI (we use Augment).
My role is to be sceptical and I’m proud of that, so my fundamental question was “Why?”, “What problem are you trying to solve?”. The response was “Its the way the industry is going?”….I couldn’t accept that as a valid reason. So I had many a row with my CTO about the fact it was directionless. We didn’t just have Devs playing with Augment, but product and in fact anyone who said they were willing. However, I kept pressing because I always prefaced my pushback is “My job is to focus on quality right? I’m saying I see a quality risk with where this is going. My job is to explain why I see that, but you don’t seem to want to hear it”.
Now I wasn’t against Agentic AI at all, I was against the motivation behind it, not the tooling. Ultimately, what are we trying to achieve by using Agentic AI and how will we know we’ve been successful? If we were talking about any other tool that involved a license cost, we would have had to put a case for buying it and what the benefit to the business would be. But the CTO just bought it and told people to use it. No cost/benefit case necessary!
So 6 months in this is what I established:
- The developers had very different adoption. Some didn’t use it because it wasn’t great with C++, some used it but were cautious where and some were all in - almost vibe coding.
- The sprints of the high adopting teams slowed down - for exactly my concerns around quality. They could change code quickly, but when we tested it we would find flaws quickly - some bugs were so obvious that our reaction would be “hang on, why didn’t you see this bug when you coded it?”. The problem was exaggerated by the turn around time to fix the issues, because the devs were trying to use Augment to do the fixes.
- The devs on rare occasions when using augment could be guilty of scope creep. I.e. if they were working on a bit of code for a ticket and there was an underlying bit of tech debt they never got around to, they might ask augment to look at it. So there was an impact to quality at times there that again, slowed us down
- Those outside engineering where using augment for analysis, prototyping and workarounds. Useful no doubt, but nothing directly improving the engineering throughput
So in summary, my job is to be sceptical and focus on quality. I’m accepting that the AI agents are here to stay but I’m highlighting the risk to quality in how we are adopting it. I’m showing clear metrics that we are not actually getting faster, we’re getting slower and I can prove that QA/QE are not the blocker in this. So I’m showing my CTO, that their implied goal of “getting faster” isn’t happening. Hope that helps 