🤖 Day 10: Critically Analyse AI-Generated Tests

📆 30 Days of Testing
,
22

Hey All,
For this challenge, I used Chatgpt to create the feature description and Gemini to create test scenarios

What I did here

  1. I first tried with something like FB’s registration feature, here is the chat:
    https://g.co/gemini/share/e629a1c5d170

  2. I tried something a little more complicated, and chose the segmentation feature for an AB testing tool for this one I am currently interested in this and here is the result of this chat: https://g.co/gemini/share/65bde2ccc016

Findings:

How do the AI-generated tests differ from tests you would have created?

  1. The writing format itself is not that different but coverage is missing some points that I would have added anyway even if it was not explicitly written in feature des. i.e. OAuth integration sceanrios for fb’s alternative signup option

  2. Did not like the scenarios for the segmentation feature at all but maybe a different prompt or tool might have done a better job here, either way can use it as a guidline

Your perspective on how AI-generated tests benefit the testing process.
facilitates the boring bits maybe, will definitely use it for repetitve tasks and maybe fore new features coverage if reviewing the scenarios will take less time than writing them

What do you see as the limitations of the tool you evaluated?
scenarios for a more complex feature was taken too out of context, need to evalute this some more

2 Likes