In a scenario where you have an API spec or a series of API specs what AI tooling or GPT prompt would you recommend to generate tests please?
Many thanks
3 Likes
My core technical strength lies in API Automation and testing and based on my experience over the years i haven’t seen a single tool can generate complete test coverage just from API Spec.
Most AI/GPT tools tend to generate tests directly from JSON Contract basically validating each field and structure. That’s help[ful but it doesn’t automatically translate into real functional coverage or business level validation.
What has worked better for me is building a test matrix first -mapping existing scenarios , business flows, edge cases and negative conditions agaisnt the API Contract. Once the mapping exist, AI tools can help generate boundary level validations, Schema checks & some functional test scaffolding more effectively.
Without the structural matrix, its easy to miss real world integration scenarios even if the JSON contract looks fully covered.
2 Likes
Curious, as someone who manually generates the tests from what I know are correct flows against API’s. I want to know how on earth an AI tool can know what to test because describing the flows is probably a 200 page document. How do we tell an AI to generate tests if an API has around 30 methods but very specific flows? how do you even explain and ask it to look into all the corner cases when the data itself defines some of the legal flow activities and thus defines when negative tests are in fact negative. Does it know when a test is a negative test or does it just implement tests for all sane possibles? Howe long does it really take to learn to use.
I once learned (wasted) 6 months on a GUI automation tool, we persisted with it for a year, and in the end all the tool taught us is where our workflow was just plain confusing for users, what is the practical experience and outcomes for early bug detection of yet another “power-tool” in reality?
1 Like