Alright, this is going to be a long post so get ready! In the past I had a project going on to do everything QA-related for 6 months with AI. The first few months were absolute sh*t but when my descriptions of AI agents became PAGES and not sentences it was magnificent.
Iâm not saying itâs the holy grail and AI the the best but I felt like a Dev Tech Lead, which job consists of 80% code reviews and 20% writing stuff himself. That was me⌠AI made 80% of the output and I added 20% more.
Remember this project was ~6months at first it was probably the other way around, I added 80% and AI 20% but overtime when the models learned more about my project and way of working and my description of AI agents became larger and more specified it became more and more accurate.
Itâs like a blender, put nice ingredients in it and youâll get a smoothie.
Put poop in it, well itâs still poop after mixing just liquid 
.
Iâm hoping to find new ideas and suggestions from you guys that I can try out or maybe even give you guys some ideas of what AI could be used for.
I have a new internal project at my current employer and Iâm going to recreate the same process only bigger and better! Iâll explain what I did/have in mind.
So everything on the schemaâs below:
- Green is manual work
- Blue is AI work + it has an individual AI agent which is specifically trained for that purpose and that purpose ONLY.
Reminder: EVERYTHING is AUTOMATED and looped until the AI Agent says âThatâs good, I cannot think of anything newâ. + I still did the 20% Job of adding things myself on each BLUE block (then it would loop again through the process)
Step 1:
We have a manual input of business & functional analysis.
After itâs submitted the first AI Agent will review the analysis and produce questions for the analyst to remove assumptions. Together at the same time 2 other agents will start producing:
- requirements and acceptance criteria
- After writing REQ & AC there is another agent which reviews this output adds new REQ & AC if required and pushes it towards the functional analysis and looped into a review again until all AI Agents say âThatâs good, I cannot think of anything newâ
Extra explanation: You can see the grey ish background here. Which basically means that all the AI agents in that background are linked to each other, meaning theyâll loop and refactor Automatically based on an Update in the previous state (manual or by AI). This is for all following schemaâs. We created scripts that took the output from the AI and added it to the Analysis in this picture.
Step 2:
Based of those requirements, analysis and Acceptance Criteria my agents will create Test scenarioâs/cases/steps. I had issues here with coverage so I had to create another AI Agent which specifically thought of Edge Cases.
Step 3:
This was only added later towards the project whenever we had enough data from JIRA/Developers. We on purposely made mistakes letâs say developer X didnât add ârequired fieldsâ often, it would write this as output as âriskâ that and it would advise on it.
The advise would be: Give this user story to developer Y to produce the least bugs.
Or if Developer X picks up this ticket, donât forget to check this or donât forget to add required fields. (If I remember correct, it was about 82% accurate)
Step 4:
We made it generate code for the devs according to guidelines & standards, and already name Locators, add IDs for automation, API Specs etc. Which was really great! It was reviewed in a loop.
Step 5a:
Based off step 4 we were able to create POM files for automation, create test scripts with Cypress back in the days and review test coverage (compared to REQ & AC)
There was also a trigger: if the code changed, the POM files Agent validated it and perhaps created more content, triggering the rest of the agents again etc etc âŚ
Step 5b:
Parallel we had an AI agent creating performance testing scripts, in JMeter. Which was âhardâ to get it right, the description was probably the longest because in our analysis we had to get a specific section for performance testing with all the proper requirements.
Step 6:
I donât have to explain this much I suppose, we created files/scripts for Azure DevOps pipelines.
This was very accurate, like almost 100% since over time our description of the agent was pages long 
Step 7:
The reporting section! Here is still a lot to improve imho. Whenever the automated tests were executed OR manual tests. (Automated for fetched from Azure DevOps, manual had to still be manually uploaded, I didnât finish this part yet)
It analysed like:
- Repeating failing tests and advised on it
- Noted down pass/fail %'s
- Spotted flaky tests (that we made on purpose)
For performance testing it did the same, found bottlenecks etc and advised on how to approach to debug. Same for Anomaly detection
We donât like reporting so we though, letâs make a powerpoint
So we had an Agent to create graphs specifically for this.
Which were fed to the Agent to create powerpoints (but they were ugly af)
Step 8:
This is a âside projectâ where we got Agents for our âshift rightâ
It scanned logs, activity etc and reported it
For example when we launched a vulnerability scan with loads of attacks, it will trigger alerts of a hacker being present and required attention.
Anomaly detection was for example: When in a wizard/flow people (me duh) would bypass a step via API and it would detect that itâs not the correct flow and report that people are âtrying to cheatâ
This wasnât too mature either, I wanted to add way more into the log reviews etc.
BUT the nicest part of this was that if there were bugs detected in production, it would automatically create test cases for it if it was forgotten! (that was the initial goal)
So this is basically what I did with AI at a client, I donât have IP of it nor access anymore 
but Iâm going to recreate it for an internal project at my current employer.
PS: I didnât set this up all alone duh! I had help from a data guy who created me all these agents and together we created the linking process and I provided it training data. (internal documents, videoâs, blogs, etc )
It was all run on a server, not in the cloud.
What Iâm looking for is new ideas to add onto it to try it out for âfunâ since itâs an internal project and we are trying to discover what AI can do. I am now aware that it will probably require at least 6 months to get some decent results so I have patience 
Things I want to add already:
- Threat Modeling
- Security - SAST
- Add BDD
- I also wanted to add some capacity planning for testers
Thanks for reading and hope you got some ideas or might give me some ideas!
