- Yes
- No
- Didnāt know you could!
0
voters
Bonus points for sharing how you do it
4 Likes
Yea! So Iāve been recently using a few (multiple, each their own purpose) AI Agents for testing purposes and one of them is for planning.
I can feed to user stories and it would give me a list of user stories which could go into the sprint based on priority and story points. Further then that, it would also tell me which tester of my team would best be fit to pick up the user story based on their expertise and logged bugs in the past (that Iāve fed to the Agent).
Nice to know, we keep our holidays and office days in an online excel sheet, which our agent has access to, and before asking the agent something like this (like before a sprint planning etc) Iāll retrain it with the links, blogs, videoās connected to the Agent, so heās up to date with our holidays etc!)
Works out pretty good, Iāll always have to review it but better then starting to write from scratch and sometimes people ask ācan I do this?ā and Iāll manually swap it out but coming to think of it, I can just ask our agent this also next timeā¦ (only happened once so far)
My agent also makes a workflow for me for the original test planning.
3 Likes
Two things/concerns that immediately came to mind here:
- Is this an Agent that youāve spun up internally, or a publicly available one (i.e. that may use the data you feed it for further training)? The former should be fine, but depending on the work youāre doing (my current project is a government contract) that second choice could be very questionable from a data security perspective. Even a user story without PII could still be an insight into commercial secrets.
- How have you verified the accuracy of the recommendations that the agent gives you? Is it making the same decisions that you would have, if you took the time? Iām also curious about what aspects of a previously logged bug makes a user suitable to pick up a new story.
Full disclosure, Iām very much an AI sceptic, at least for the current models being floated around as must-have tools, that are usually LLMās or some other version of Generative AI (which cannot reason, they can only provide convincing guesses based on probabilities).
2 Likes
Aah good questions and obvious ones! 
So to give a bit more context, weāve been on an āAI in QAā track for a while. Iām also a big sceptic about it. But from what Iāve been using for some things itās quite nice and for some itās really bad 
But nothing is publicly available, we also donāt share anything with has confidential information (secrets etc) just to be sure.
Itās basically a manual review.
Iāve said it to my Coās this way:
You know how in a big team there is a āLead Devā who does 20% dev work but also 80% Code Reviews? Well using AI in QA for āeverythingā is the same. You adjust 20% yourself and the rest is just " AI Review".
A few years ago we made a model with tensorflow that could tell us by ~82% accuracy (over time) where bugs would occur based on which developer was going to pick up the user story. Based on their experience and bugs logged to their User Stories, the model could prediction defects (such as, āperson X will forget to make this field mandatoryā) and that would be useful information for that developer to increase their maturity.
Iām gonna say, in order to step into this process you really need to turn around everything in your organization, everything needs to be in an āexactā way to work. (for now)
Exactly. Itās still sort of guess work and the answer will not be 100% correct. Hence the āfull time reviewā job, itās just something weāve been experimenting with since Jan 2024. I kind of hate it XD (but I do admit, itās quite nice in some cases)
2 Likes
Good clarifications overall, cheers.
Iām not sure I still quite understand where the data is going sorry - is the agent something in house that never can send anything to a 3rd party outside your company? Or is that still a risk vector?
I may need to clarify what I meant by business secrets - it doesnāt have to be āconfidentialā information in the way that many people think. Business processes alone (which are what youāre going to feed into the agent via user stories) can often be considered commercial secrets, from a legal standpoint.
2 Likes
My bad, itās build internally.
Since itās internally build the business processes donāt matter much. We are a consultancy company, so itās not like we have a webshop or anything alike or have secret flows to connect to a bank or w/e. XD
Most of it are todoās or stuff for our internal application. Nothing to fancy 
2 Likes
Iām quite new in this field, so Iām not that experienced, but atm I try to generate testcases by using AI. But this needs much more refinement to be that helpful :X
4 Likes
I find the paid version fairly useful and it remembers the train of thought to some extent.
I am a light usage user, like a very average buddy to bounce things off or a short cut to many things.
It has access to a lot more knowledge and data than most individuals would have, yes some of it is flawed and you need to practice to ask the right questions but that broad data and information access is useful.
For the above, I would not put in the actual intelligent class, almost all of the above I can get in other ways, in many times better more accurate ways like talking to someone but I still see value in it and convenient.
You can ask for examples, risk ideas, test ideas, formatting or giving it problems to give ideas to help you solve. Note I use the term ideas a lot when using it, it does not solve my problems it gives me fallible insight and ideas to help me solve my problems.
Itās not just planning, many aspects in testing it can give ideas, Iāve been using a bit more recently with automation for example. Because Iām using a new tool and Iām like a junior user in that tool its average responses are still new insight for me.
I still prefer human mentors or human sounding boards but this can be a convenient semi-substitute but will access to a much larger knowledge base.
2 Likes
I mostly use LLMās and as of recently Testim (from Tricentis) for UI scripting.
2 Likes
Very useful idea. How is AI be able to access to your online excel sheet and how secure is it?
Itās just a link you provide it and it can read the tables.
Itās just like providing it a link of a blog or youtube video and itās able to process it. (I donāt know the magic behind AI on āhowā exactly)
You could also manually upload excel files but then itās not āliveā
As I canāt link our own tool; I did some research before hand and ātaskadeā compares a lot to it. It has premium features where you can link your data to it, for example the blog about Cypress & POM in the sc. (Only downpart here is that itās in their system and now āyoursā)
But as you can see, you can add a lot of different data to it & integrations.
Sc of Taskade:
2 Likes
Also, there a way to upload your data source for paid ChatGPT.
Itās called custom GTPs Explore GPTs.
You can upload your data and make a prompts to make some test planing or coverage.
I wouldnāt say itās accurate right how however itās a matter of further exploring and making correct prompt to get best answers.
