This wiki started at TestBash Spring 2023 during a panel discussion called “How AI Can Impact Our Testing Roles”. Feel free to edit and add new information and keep it up to date. Links to company/promotional material will be removed. For example, if adding a blog link, please only share a personal blog. Just look for the Edit button. It looks like this:
Why should we care about AI in relation to the testing craft?
Provide compelling reasons why we need to care about AI and its impact on the testing craft
- Within several years most probably it will be everywhere and using AI will be an essential skill
- It will help you to be more productive
Examples of how AI can assist testing
Provide real-life examples of how AI has assisted your testing.
- Generate test cases
- Build end-to-end tests
- Build unit tests
- Helps to prioritize and select which tests to execute for a feature change
- Generate some or all of the code of a test case
- Significantly reduce test maintenance for end-to-end tests
Risks, worries and concerns about AI in relation to the testing craft
Capture any risks, worries and concerns about AI, the ethics of AI and its impact on the testing craft
- There are many risks. Verified sources of data and unconscious bias in test data selection are major risks. These may result in lower quality if a system is tested using incorrect data and/or only a subset of possible data coverage. — Charmaine Short (source)
- AI Models are sill in learning phase, at a big pic. In the linear algebraic\computational mathematical models we use, there may be delta error now, which later, at extreme level of calculations\optimizations which may end up in major error with impact to business. — Roopam Chopra (source)
- Bias — Ville Rytinki (source)
- add more
Ongoing conversations about AI and testing
- How will ChatGPT change the testing industry?
- What part of your job would you give to AI to test faster?
- I wonder in which activities could ChatGPT help you in your testing?
- AI applications in testing
- Artificial Intelligence in Testing (AIiT)
- What is the license of code generated from CHAT GPT (or Generative AI)?
- add more
Helpful tools that use AI to assist testing efforts
AI-assisted test tool vendors are welcome to add their tools to this list.
- testRigor uses GPT-4 to build working end-to-end test based on plain English test case description
- aqua generates test cases from a requirement or auto-creates test steps from a description
- diffblue generates unit test cases from the given codebase
- MagnifAI helps with visual testing
- Codium “By analyzing your code, docstring, and comments, and by interacting with you, TestGPT suggests tests as you code.”
- ScopeMaster uses AI to automate test generation from user stories. It also tests the user stories (like Sonarqube but for requirements)
- Socket AI – Socket is using ChatGPT to examine every npm and PyPI package for security issues.
- ChatGpt Test Case Creator by @guilhermevigneron
- SofySense – advanced insights, analysis, and assistance for all your QA needs using the power of OpenAI.
- SeaLights Helps to prioritize test execution
- Katalon - provide a jira plugin that will read user stories and generate test cases.
Books on AI
- Artificial Intelligence and Software Testing
- Testing in the digital age
- Not with a Bug, but with a Sticker: Attacks on Machine Learning Systems and What To Do About Them
- How To Test a Time Machine by Noemi Ferrera - great chapters on using AI-related tools for testing, and testing AI apps and tools.
Videos on AI
- AI Features for Software Testers
- Generative AI in Software Testing
- Prompt Engineering in Software Testing
- Will openAI Change Software Testing?
- AI generated software testing book, is it good?
- 10 AI-powered software testing tools
- An Overview of Machine Learning for Testers by Paul Maxwell-Walters
- Current State of AI Features for Software Testers by @dnlknott
Articles on AI
- A brief evaluation of ChatGPT-4 quality
- 3 Reasons AI Won’t Save You Yet by @tara.walton
- My collection of +20 Chat-GPT Prompts For Quality Assurance by @vincentferreira
- Where does ‘AI Assistance’ start, exactly? by @whitenoise
- How should we view AI as testers? by @sjwatsonuk
- AI Testing – Measures and Scores, Part 1 by Jeff Nyman
- Can Generative AI Write Good Exploratory Tests? by @cakehurstryan
- The Ethical Considerations in AI-Driven Test Automation by @pricilla09
- A Framework for Using Generative AI in Software Testing by John Kinnebrew
- Prompt Engineering Tools & Libraries
- Exploring Generative AI in Software Development and Testing by Ron Wilson