This wiki started at TestBash Spring 2023 during a panel discussion called “How AI Can Impact Our Testing Roles”. Feel free to edit and add new information and keep it up to date. Links to company/promotional material will be removed. For example, if adding a blog link, please only share a personal blog. Just look for the Edit button. It looks like this:
Why should we care about AI in relation to the testing craft?
Provide compelling reasons why we need to care about AI and its impact on the testing craft
- Reason 1
- Reason 2
Examples of how AI can assist testing
Provide real-life examples of how AI has assisted your testing.
- Example
- Example
Risks, worries and concerns about AI in relation to the testing craft
Capture any risks, worries and concerns about AI, the ethics of AI and its impact on the testing craft
- There are many risks. Verified sources of data and unconscious bias in test data selection are major risks. These may result in lower quality if a system is tested using incorrect data and/or only a subset of possible data coverage. — Charmaine Short (source)
- AI Models are sill in learning phase, at a big pic. In the linear algebraic\computational mathematical models we use, there may be delta error now, which later, at extreme level of calculations\optimizations which may end up in major error with impact to business. — Roopam Chopra (source)
- Bias — Ville Rytinki (source)
- add more
Ongoing conversations about AI and testing
- How will ChatGPT change the testing industry?
- What part of your job would you give to AI to test faster?
- I wonder in which activities could ChatGPT help you in your testing?
- AI applications in testing
- Artificial Intelligence in Testing (AIiT)
- What is the license of code generated from CHAT GPT (or Generative AI)?
- add more
Helpful tools that use AI to assist testing efforts
AI-assisted test tool vendors are welcome to add their tools to this list.
- SeaLights
- testRigor uses GPT-4 to generate test cases based on the description
- aqua generates test cases from a requirement or auto-creates test steps from a description
- diffblue generates unit test cases from the given codebase
- MagnifAI helps with visual testing
- Codium “By analyzing your code, docstring, and comments, and by interacting with you, TestGPT suggests tests as you code.”
- ScopeMaster uses AI to automate test generation from user stories. It also tests the user stories (like Sonarqube but for requirements)
- Socket AI – Socket is using ChatGPT to examine every npm and PyPI package for security issues.
- ChatGpt Test Case Creator by @guilhermevigneron
- SofySense – advanced insights, analysis, and assistance for all your QA needs using the power of OpenAI.
Books on AI
- Testing in the digital age
- Not with a Bug, but with a Sticker: Attacks on Machine Learning Systems and What To Do About Them
Videos on AI
- Will openAI Change Software Testing?
- AI generated software testing book, is it good?
- 10 AI-powered software testing tools
Articles on AI
- A brief evaluation of ChatGPT-4 quality
- 3 Reasons AI Won’t Save You Yet by @tara.walton
- My collection of +20 Chat-GPT Prompts For Quality Assurance by @vincentferreira
- Where does ‘AI Assistance’ start, exactly? by @whitenoise