# AI and Testing wiki **URL:** \ **Category:** Wikis\ **Tags:** machine-learning, ai, wiki\ **Created:** [23 March 2023 17:00 UTC](https://club.ministryoftesting.com/t/ai-and-testing-wiki/66134 "2023-03-23T17:00:01Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![simon\_tomes](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/simon_tomes/32/14009_2.png) [@simon\_tomes](https://club.ministryoftesting.com/u/simon_tomes)\ **Post date:** [23 March 2023 17:00 UTC](https://club.ministryoftesting.com/t/ai-and-testing-wiki/66134/1 "2023-03-23T17:00:01Z")
_This wiki started at TestBash Spring 2023 during a panel discussion called “[How AI Can Impact Our Testing Roles](https://www.ministryoftesting.com/testbash-talks/bcdb383f)”. Feel free to edit and add new information and keep it up to date. Links to company/promotional material will be removed. For example, if adding a blog link, please only share a personal blog. Just look for the Edit button. It looks like this:_ ![Screenshot 2023-03-29 at 12.18.59](https://us1.discourse-cdn.com/flex020/uploads/ministryoftesting/original/2X/a/a19c3bdad7c4426b35b16ba1fe4b3f01f38fd9b8.png) ## Why should we care about AI in relation to the testing craft? _Provide compelling reasons why we need to care about AI and its impact on the testing craft_ 1. Within several years most probably it will be everywhere and using AI will be an essential skill 2. It will help you to be more productive ## Examples of how AI can assist testing _Provide real-life examples of how AI has assisted your testing._ - Generate test cases - Build end-to-end tests - Build unit tests - Helps to prioritize and select which tests to execute for a feature change - Generate some or all of the code of a test case - Significantly reduce test maintenance for end-to-end tests ## Risks, worries and concerns about AI in relation to the testing craft _Capture any risks, worries and concerns about AI, the ethics of AI and its impact on the testing craft_ - There are many risks. Verified sources of data and unconscious bias in test data selection are major risks. These may result in lower quality if a system is tested using incorrect data and/or only a subset of possible data coverage. — Charmaine Short ([source](https://www.linkedin.com/posts/ministry-of-testing_what-are-the-risks-of-using-ai-in-testing-activity-7043953652960501760-fLlX?utm_source=share&utm_medium=member_desktop)) - AI Models are sill in learning phase, at a big pic. In the linear algebraic\computational mathematical models we use, there may be delta error now, which later, at extreme level of calculations\optimizations which may end up in major error with impact to business. — Roopam Chopra ([source](https://www.linkedin.com/posts/ministry-of-testing_what-are-the-risks-of-using-ai-in-testing-activity-7043953652960501760-fLlX?utm_source=share&utm_medium=member_desktop)) - Bias — Ville Rytinki ([source](https://www.linkedin.com/posts/ministry-of-testing_what-are-the-risks-of-using-ai-in-testing-activity-7043953652960501760-fLlX?utm_source=share&utm_medium=member_desktop)) - _add more_ ## Ongoing conversations about AI and testing - [How will ChatGPT change the testing industry?](https://club.ministryoftesting.com/t/how-will-chatgpt-change-the-testing-industry/64010) - [What part of your job would you give to AI to test faster?](https://club.ministryoftesting.com/t/what-part-of-your-job-would-you-give-to-ai-to-test-faster/65359) - [I wonder in which activities could ChatGPT help you in your testing?](https://club.ministryoftesting.com/t/i-wonder-in-which-activities-could-chatgpt-help-you-in-your-testing/64639) - [AI applications in testing](https://club.ministryoftesting.com/t/ai-applications-in-testing/43522) - [Artificial Intelligence in Testing (AIiT)](https://club.ministryoftesting.com/t/artificial-intelligence-in-testing-aiit/64428) - [What is the license of code generated from CHAT GPT (or Generative AI)?](https://club.ministryoftesting.com/t/what-is-the-license-of-code-generated-from-chat-gpt-or-generative-ai/66333) - _add more_ ## Helpful tools that use AI to assist testing efforts _AI-assisted test tool vendors are welcome to add their tools to this list._ - [testRigor](http://testrigor.com/) uses GPT-4 to build working end-to-end test based on plain English test case description - [aqua](https://aqua-cloud.io/) generates test cases from a requirement or auto-creates test steps from a description - [diffblue](https://www.diffblue.com/) generates unit test cases from the given codebase - [MagnifAI](https://magnif.ai/) helps with visual testing - [Codium](https://www.codium.ai/) “By analyzing your code, docstring, and comments, and by interacting with you, TestGPT suggests tests as you code.” - [ScopeMaster](https://www.scopemaster.com/) uses AI to automate test generation from user stories. It also tests the user stories (like Sonarqube but for requirements) - [Socket AI](https://socket.dev/blog/introducing-socket-ai-chatgpt-powered-threat-analysis) – Socket is using ChatGPT to examine every npm and PyPI package for security issues. - [ChatGpt Test Case Creator](https://github.com/guilhermevigneron/Chatgpt_Test_Case_Creator) by @guilhermevigneron - [SofySense](https://sofy.ai/products/sofysense/) – advanced insights, analysis, and assistance for all your QA needs using the power of OpenAI. - [SeaLights](https://www.sealights.io/) Helps to prioritize test execution - [Katalon](https://katalon.com/) - provide a jira plugin that will read user stories and generate test cases. ## Books on AI - [Artificial Intelligence and Software Testing](https://shop.bcs.org/store/221/detail/workgroup?id=3-221-9781780175768) - [Testing in the digital age](https://www.ict-books.com/topics/digitaltesting-en-info) - [Not with a Bug, but with a Sticker: Attacks on Machine Learning Systems and What To Do About Them](https://www.amazon.com/Not-Bug-But-Sticker-Learning/dp/1119883989) - [How To Test a Time Machine by Noemi Ferrera](https://www.packtpub.com/product/how-to-test-a-time-machine/9781801817028) - great chapters on using AI-related tools for testing, and testing AI apps and tools. ## Videos on AI - [AI Features for Software Testers](https://www.youtube.com/watch?v=2RO1ZG_0T2k) - [Generative AI in Software Testing](https://youtu.be/rZEmxEmwFvs) - [Prompt Engineering in Software Testing](https://youtu.be/8-LcECT4jjQ) - [Will openAI Change Software Testing?](https://www.youtube.com/watch?v=oC3CxOXQjEo) - [AI generated software testing book, is it good?](https://www.youtube.com/watch?v=JoxWyOq-kbA) - [10 AI-powered software testing tools](https://www.youtube.com/watch?v=Ow4dNxqZYlY) - [An Overview of Machine Learning for Testers](https://www.youtube.com/live/nzZxvf6JLCc?t=120&si=VQiI4KR-TgRtXdFH) by Paul Maxwell-Walters - [Current State of AI Features for Software Testers](https://www.youtube.com/watch?v=2RO1ZG_0T2k) by @dnlknott - [What is GitHub Copilot | Documentation| Code Generation| Optimisation | Bug fixing](https://www.youtube.com/watch?v=nF4-nCt0B6A&list=PLUlzmYT8n_5utiy_i2R_sAlOWLVji0c8g) by @gaurav.jaz ## Articles on AI - [A brief evaluation of ChatGPT-4 quality](https://qase.io/blog/chatgpt-4/) - [3 Reasons AI Won’t Save You Yet](https://medium.com/@tara.a.walton/3-reasons-ai-wont-save-you-yet-dc0e0225dc27) by @tara.walton - [My collection of +20 Chat-GPT Prompts For Quality Assurance](https://medium.com/@vincent.ferreira/my-collection-of-20-chat-gpt-prompts-for-quality-assurance-7eaef6633bd5) by @vincentferreira - [Where does ‘AI Assistance’ start, exactly?](https://medium.com/@Superwhitenoise/where-does-ai-assistance-start-exactly-cef6735c655a) by @whitenoise - [How should we view AI as testers?](https://stevethedoc.wordpress.com/2023/06/18/how-should-we-view-ai-as-testers/) by @sjwatsonuk - [AI Testing – Measures and Scores, Part 1](https://testerstories.com/2023/07/ai-testing-measures-and-scores-part-1/) by Jeff Nyman - [Can Generative AI Write Good Exploratory Tests?](https://cakehurstryan.com/2023/07/03/can-generative-ai-write-good-exploratory-tests) by @cakehurstryan - [The Ethical Considerations in AI-Driven Test Automation](https://www.lambdatest.com/blog/ethical-considerations-in-ai-driven-test-automation/) by @pricilla09 - [A Framework for Using Generative AI in Software Testing](https://www.mabl.com/blog/a-framework-for-using-generative-ai-in-software-testing) by John Kinnebrew - [Prompt Engineering Tools & Libraries](https://www.promptingguide.ai/tools) - [Exploring Generative AI in Software Development and Testing](http://qarevolution.com/exploring-generative-ai-in-software-development-and-testing/) by Ron Wilson ## Podcasts - [Breaking Down Today’s Machine Learning Technology with Christina Pawlikowski](https://podcasts.bcast.fm/e/4n90rr98) ## Newsletters on AI - [The AI Product Report](https://www.productreport.ai/) - [Futurepedia](https://www.futurepedia.io/newsletter) ## AI Industry news - [OpenAI’s board had safety concerns. Big Tech obliterated them in 48 hours](https://www.latimes.com/business/technology/story/2023-11-20/column-openais-board-had-safety-concerns-big-tech-obliterated-them-in-48-hours) ---
**Author:** ![mahatheed](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/mahatheed/32/10871_2.png) [@mahatheed](https://club.ministryoftesting.com/u/mahatheed)\ **Post date:** [31 October 2023 15:50 UTC](https://club.ministryoftesting.com/t/ai-and-testing-wiki/66134/6 "2023-10-31T15:50:03Z")
Wow that’s an amazing collection of different sources to learn. Thanks for collating it. ---
**Author:** ![contemplation101](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/contemplation101/32/12515_2.png) [@contemplation101](https://club.ministryoftesting.com/u/contemplation101)\ **Post date:** [12 November 2023 12:49 UTC](https://club.ministryoftesting.com/t/ai-and-testing-wiki/66134/7 "2023-11-12T12:49:09Z")
In a research project with University of Texas Dallas we have developed a MVP with these key features: 1. Prompt library to customize inputs for each testing need. 2. Integrates 8 different LLMs to provide a range of capabilities. 3. Optimized for generating software testing artifacts and content. 4. Follows a client-server model for accessibility and scalability. 5. Outputs can feed directly into testing and QA workflows. 6. Enables iterative drafting and refining of test plans/reports. 7. Designed to integrate with agile development environments. 8. Improves velocity and quality of test documentation. To recap, this software solution aims to enhance agile testing processes specifically by using its #promptdesign library and multi-LLM design to produce high-quality test plans, reports, and other QAtesting documentation more efficiently. The multi-LLM solution provides the flexibility and optimization to accelerate testing activity in agile environments. Send me a note if you would like to learn more, I’m a new user and cannot attach the design documentation. The [multi-LLM Harvey Ball analysis is available on LinkedIn here](https://www.linkedin.com/feed/update/urn:li:activity:7129441184057327616?utm_source=share&utm_medium=member_desktop). ---
**Author:** ![kinofrost](https://avatars.discourse-cdn.com/v4/letter/k/bcef8e/32.png) [@kinofrost](https://club.ministryoftesting.com/u/kinofrost)\ **Post date:** [12 November 2023 15:33 UTC](https://club.ministryoftesting.com/t/ai-and-testing-wiki/66134/8 "2023-11-12T15:33:41Z")
AI is a magician performing a trick. You don’t know how it works, you make assumptions, you’re amazed at the output. And the magician never reveals their secrets. Always be careful about solutions you don’t understand, especially if you don’t understand the problem. The responsibility stops with you. [You’re the one who ends up in the news.](https://www.reuters.com/legal/new-york-lawyers-sanctioned-using-fake-chatgpt-cases-legal-brief-2023-06-22/) ---
**Author:** ![system](https://sea2.discourse-cdn.com/flex020/user_avatar/club.ministryoftesting.com/system/32/14378_2.png) [@system](https://club.ministryoftesting.com/u/system)\ **Post date:** [22 March 2025 17:00 UTC](https://club.ministryoftesting.com/t/ai-and-testing-wiki/66134/9 "2025-03-22T17:00:10Z")
This topic was automatically closed after 730 days. New replies are no longer allowed.