AI and Testing wiki

This wiki started at TestBash Spring 2023 during a panel discussion called “How AI Can Impact Our Testing Roles”. Feel free to edit and add new information and keep it up to date. Links to company/promotional material will be removed. For example, if adding a blog link, please only share a personal blog. Just look for the Edit button. It looks like this:
Screenshot 2023-03-29 at 12.18.59

Why should we care about AI in relation to the testing craft?

Provide compelling reasons why we need to care about AI and its impact on the testing craft

  1. Reason 1
  2. Reason 2

Examples of how AI can assist testing

Provide real-life examples of how AI has assisted your testing.

  • Example
  • Example

Risks, worries and concerns about AI in relation to the testing craft

Capture any risks, worries and concerns about AI, the ethics of AI and its impact on the testing craft

  • There are many risks. Verified sources of data and unconscious bias in test data selection are major risks. These may result in lower quality if a system is tested using incorrect data and/or only a subset of possible data coverage. — Charmaine Short (source)
  • AI Models are sill in learning phase, at a big pic. In the linear algebraic\computational mathematical models we use, there may be delta error now, which later, at extreme level of calculations\optimizations which may end up in major error with impact to business. — Roopam Chopra (source)
  • Bias — Ville Rytinki (source)
  • add more

Ongoing conversations about AI and testing

Helpful tools that use AI to assist testing efforts

AI-assisted test tool vendors are welcome to add their tools to this list.

  • SeaLights
  • testRigor uses GPT-4 to generate test cases based on the description
  • aqua generates test cases from a requirement or auto-creates test steps from a description
  • diffblue generates unit test cases from the given codebase
  • MagnifAI helps with visual testing
  • Codium “By analyzing your code, docstring, and comments, and by interacting with you, TestGPT suggests tests as you code.”
  • ScopeMaster uses AI to automate test generation from user stories. It also tests the user stories (like Sonarqube but for requirements)
  • Socket AI – Socket is using ChatGPT to examine every npm and PyPI package for security issues.
  • ChatGpt Test Case Creator by @guilhermevigneron
  • SofySense – advanced insights, analysis, and assistance for all your QA needs using the power of OpenAI.

Books on AI

Videos on AI

Articles on AI


Newsletters on AI