What heuristics can we create for AI in Testing?

🙋 Questions
, , , ,
1

Jarsto van Santen’s (@jvsanten) talk at TestBash Brighton 2024 and recent LinkedIn post got me thinking.

Can we collate a set of heuristics for AI in Testing? For example, one heuristic might be the “Trained or not?” heuristic. For example, when relying on GPT how do we check the data it was trained on and how reliable it is? How does that impact our test ideas, approach, and strategy?

Much like the classic Test Heuristics Cheat Sheet, what opportunity do we have to create an AI in Testing Heuristics Cheat Sheet?

1 Like
2

Here is a list of syndromes by @mb1 and James Bach:

2 Likes
3

In manual testing, semi-AI tools can assist by generating test scripts based on the requirements or even by analyzing the UI through AI.
A practical way to measure the quality of these AI-generated tests is by using standard metrics such as the failure rate or the number of defects found.
These metrics can serve as valuable heuristics to assess the effectiveness of the AI tools.

1 Like
4

I have collated some checkpoints (prompting wisdom) here:

Prompting Checklist for Testers - Rahul’s Testing Titbits

I can say that these are all heuristics for AI prompting / usage.

2 Likes