Are we testing AI, or is AI testing us?

Discussions
, ,
1

Lately, with GenAI and LLM tools pouring in like a flood for both development and testing, I keep wondering: are we still testing systems, or slowly being tested by the very AI systems that we are relying upon?

I’m wondering what others in the community are doing to adapt their QA strategies along this change.

Are you really using AI to do test generation, exploratory testing, or bug triage?

Have you had any issues with trust, accuracy, or outcomes that were unexpected?

Are there some wins you can share; or maybe some hard-learned lessons?

I’m really curious to know how others are managing this. Let’s teach and learn from each other!

2

Those can be useful, but if it is improperly trained or given the wrong data, results looks fine but not if you look deeper, Perhaps read blogs from Michael Bolton,

He also has some blogs about this. Just think that a student is copy paste the results of AI to an essay.

You always need critical thinking, etc

3

I think both statements are true.

The more we use it, the more we know about AI and the more problems we will know that are there

So i created a framework via a prompt and then saw mistakes done by AI. So in a way I tested it
AI-Generated Selenium Framework Mistakes (And How I Fixed Them) | by Gaurav Khurana | Jul, 2025 | Medium

Also Since AI system learn from user feedback. in ChatGPT you might have noticed it sometimes give 2 response and we select one. That is also a kind of testing done by AI with us.