Testing Skills for Agent

Question for folks:

Does anyone have testing skills that they have used to great effect? Especially recently?

Context: my team is looking at revamping their AGENTS.md file for newer models, and I wanted to see what improvements the community has made recently.

Thank you in advance!

the biggest improvement i’ve seen is making the agent return a test contract, not just code.

for each test, require it to state the risk, starting state, exact action path, oracle, final-state readback, evidence saved, and cleanup. also make it label uncertainty instead of guessing when auth, data, timing, or earlier browser state is missing.

in AGENTS.md i’d add one hard rule: a browser task is not done because clicks completed. it is done when the intended state is read back and a failure leaves enough context for another person to inspect. keep generated tests at the lowest layer that preserves the risk, and require human review when the oracle is ambiguous.

the useful regression test for the skill itself is one known-good flow and one seeded failure. check whether it identifies the first mismatch instead of only summarizing the final symptom.

Add a one-liner: “don’t make mistakes” :smiling_face_with_horns:

Just joking

latest frontier models opus 5.5 and gpt sol 6 are great at testing with limited instructions. i dont think writing verbose detailed AGENTS.md is important anymore. the AGENTS.md should ensure instructions to meet coverage, mutation numbers, contracts, API testing are mentioned clearly. Your team specific requirements and what you expect (goals) out of automated AI testing are what you should be including in AGENTS.md. Also, invest in a pre-run wherein characterization tests of sufficiently high quality are present to lock behaviour. this would help newer code generation have appropriate baselines to take reference from.