Hi,
After going through the comments and reading observations from other members of community.
For my day to day used I could think of only 2 uses of AI so far
To create test cases using ChatGPT but I have not tried it or used it for that purpose yet.
Katalon for low code automation, although we have automation testers in the team to create test framworks.
AI bots - tools like ChatGTP can help testers to figure out edge cases and expand exploratory testing. This still depend on knowedge of bussiness rules (given bots context so they can expand based on the Models they built.
Visual Validation
This is a game changer specially compared with tests that use selenium for validation (without interaction). Applitools in one of the tools Iām aware of.
Bug History
Companies can build datasets and customize Models to help identifying common bugs that keep happened before or even give insights to testers on bugs that happened elsewhere.
There are quite a number of a use cases. In fact nearly everything that we do can probably be improved with AI.
Test data - this is something that Iāve seen before but never used, largely because I wasnāt sure whether it could be used within our systemsā¦ plus the inevitable cost to try something (financial, time or both).
Automated test execution - I was vaguely aware of this but not seen it in action. Iād be interested in this as weāre sinking time into handling flakey failing tests that didnāt even need running (although Iād rather not have flakey failing tests in the first place). As we grow our suite for our new project, I can see execution times growing to become more than āa few secondsā quite quickly.
Writing tests - I learnt about this from chatting to a colleague who uses AI for development. They can get a load of unit tests written with a click of a button, and then just need to tidy them up.
Writing / improving ACs or acceptance tests: Using ChatGPT or similar seems a low cost but I still see a lot of copy-pasting, reviewing and alike.
Improving visual testing: I really like the concept of visual testing but it quickly gained a reputation within the team that I spoke to that it was just flakey.
Generating tests by integrating with Jira: I like this but then do I trust it. I suspect that it partly depends on the quality of the tickets being generated.
Most of the tools that Iāve found reference to arenāt ones we use and so far Iāve not seen enough to make me know that I definitely can pursue (given time / cost often involved).
Adaptive testing: AI can dynamically adjust testing strategies based on real-time feedback, system behavior, and user data.
Predictive analysis: AI can predict potential issues and risks based on historical data
Anomaly detection: AI algorithms can learn the typical behavior of an application and its users. When deviations from normal patterns occur, AI can identify anomalies and potential defects
Some tools I identified to assist and improve testing,
Eggplant Digital - Model-based digital twin testing approach for comprehensive coverage (thatās one feature, Iād had to research more as I am unsure what that entails exactly). Cool product name though
Sauce Labs - enable authoring, writing, and full automation of test suites
Thereās Mabl and Tricentis Testim that many here have also mentioned
A lot as been said about the use cases for AI in testing. The ones that I think hold a lot of potential for me personally would be:
Visual Regression Testing: Using AI to differentiate visual differences is more effective than the human eye. A tool that Iāve heard for visual testing is DevTools AI, and its ability to test a frontend without having to necessarily know the supporting DOM structure.
Test Selection: Time can be saved by using an 80/20 rule for effective test run coverage. By learning from past runs and analyzing incoming changes, a more robust set of tests can be selected to be run that can comprehensive validate the incoming change. I have not had experience in this realm, and will be looking forward to using tools of this use case.
I wanted to share some insights I gathered on how AI is transforming testing. Hereās a breakdown of some areas where AI is making a significant impact:
Test Automation:
Self-healing tests are becoming extremely popular, where AI tools like Katalon, Functionize, Testim, and Virtuoso automatically update test scripts in response to any code changes. This ensures test stability even as the application evolves and mitigates any manual maintenance of the the test scripts.
Test Case Generation:
AI tools such as Applitools and TestCraft analyze application behavior to generate test cases automatically. This reduces the manual testing efforts and improves the test coverage, ultimately enhancing the efficiency in our testing processes.
Defect Prediction and Analysis:
Tools like DeepCode and CodeGuru utilize AI algorithms to analyze historical defect data and code changes, predicting and preventing potential bugs. This proactive approach to defect management helps enhance software quality by addressing issues early in the development cycle.
Reflecting on these AI uses, I believe self healing tests would be particularly beneficial in our context, given how large some codebases are - with frequent changes. These tools would ensure test stability, allowing us to focus more on testing and less on script maintenance. Additionally, AI-powered test case generation tools would help improve test coverage without increasing manual effort, which is crucial for managing a large codebase efficiently.
Lastly, leveraging AI for defect prediction and analysis would enable us to identify and address issues proactively, ultimately enhancing both the streamlines and emphasizes quality of our software development processes and maintenance.
The tools generally I use for testing is ChatGPT and Blackbox.ai.
Truely havenāt explored and experimented a lot of AI tools in the market as I am still unsure how much companyās application code one should be feeding these free AI tools.
Thanks everyone for listing so many new AI tools. I am definitely making a list of those.
AI-based unit testing: Easy to update, relatively quick to do so as well.
UI testing: Can handle minor deviations better than non-AI automations.
Test Maintenance: Can update tests based on code changes to expedite
I like these uses as it will result in a reduction of time spent maintaining cases. I canāt count how many times Iāve had to dig through source just to fix a single anchor.
I have come across many amazing AI-based tools like āKatalonā,ā Test Craftā, āTricentis Toscaā and āSikuliXā etc but havenāt been able to use them yet.
This is how AI can be helpful to QAs:
It can help generate more thought on test cases for multiple test scenarios without taking much time.
I asked chatgpt to give me a few examples of how ai is used in testing. I sometimes use it for a couple of the things on the list like test generation, test case prioritization
AI is utilized in testing in various ways:
Automated Test Generation: AI algorithms can generate test cases automatically based on requirements, specifications, or historical data, improving test coverage and efficiency.
Test Case Prioritization: AI can analyze the impact and likelihood of potential software failures, helping prioritize test cases for execution, especially in regression testing scenarios.
Anomaly Detection: AI-powered tools can detect anomalies in system behavior or performance, identifying potential issues that might not be caught by traditional testing methods.
Test Data Generation: AI algorithms can generate diverse and realistic test data sets, ensuring thorough testing of various scenarios and edge cases without relying solely on manually crafted data sets.
Hello.
According to my research, here are some ways AI is applied in testing:
AI -based Testing: AI and Machine learning algorithms are used to effectively test a software product. The objective of AI-based testing is to make the testing process smarter and highly effective. with the inclusion of AI and ML in testing, logical reasoning and problem solving methods can be applied to improve the overall testing process.
Test case generation: AI can be used to generate test cases. This can help in covering more scenarios and reducing the chances of missing any critical cases.
Defect analysis: AI can be used to analyze defects. It can predict the potential areas of the software where there could be a higher possibility of defects.
Optimization of testing processes: AI can help in optimizing the testing processes by identifying the redundant test cases and ensuring maximum test coverage with the minimum set of test cases.
In terms of tools, there are four main categories of AI driven testing tools:
Differential Tools: These tools are used to identify the differences or changes in the system that need to be tested.
Visual AI testing Tools: These tools use AI to visually validate the user interface and ensure it appears correctly to the users.
Declarative Tools: These tools allow the testers to declare what they want to test, and the tool itself figures out how to create the test.
Self healing Tools: These tolls automatically modify the test script when there are changes in the system under test.
As a software tester the testing features that would benefit me the most are those with numbers 1 to 4 in the text above.
In conclusion, the use of AI in testing can greatly enhance the testing process by making it more intelligent, efficient, and effective. It can help in delivering a high-quality product by ensuring thorough testing and reducing the chances of any critical defects slipping into the product environment.
Instead of choosing an article i skimmed through the responses and here is my analysis
AI in Software Testing: AI is increasingly integrated into software testing to automate tasks, improve accuracy, and optimize test processes. Common applications include test automation, visual testing, defect prediction, and performance testing.
Key AI Uses in Testing:
Self-Healing Tests: Tools like Katalon Studio and Functionize update test scripts based on code changes, reducing manual maintenance.
Test Case Generation: AI can create test cases from requirements or other inputs.
Visual Testing: Tools like Applitools and Percy use AI for UI consistency checks and visual validation.
Defect Prediction: AI identifies potential defects by analyzing code changes and historical data.
Performance Testing: AI simulates user behavior to optimize application performance.
Popular AI-Driven Testing Tools:
Katalon Studio: Comprehensive automation across various platforms (web, mobile, etc.).
Applitools Eyes: Detects visual changes across different environments.
Testim: Offers self-healing and intelligent test case generation.
Functionize: Integrates AI for self-healing and predictive analytics.
BlazeMeter: AI for performance profiling and optimization.
DeepCode: AI-driven static code analysis tool.
Challenges with AI Tools:
While AI can streamline processes and improve efficiency, it may not be suitable for all scenarios, especially those requiring deep contextual understanding or heuristic-based activities.
AI Limitations:
Although AI can help with many aspects of software testing, human oversight is crucial for validating AI outputs and ensuring quality.
I was interested in using AI together with mobile automation in Robot Framework. I had already used ChatGPT to create test cases from user stories and then create Robot Framework scripts from the test cases and it wasnāt too bad. Now with GPT-4o it is even better. Also directly in Qase there is a beta for generating Robot Framework scripts, but it only worked for web at the moment.
Here are some other options I got for AI and Robot Framework:
1. Visual Testing and Validation
AI can help in visual testing by ensuring that the mobile appās user interface is consistent across different devices and screen resolutions.
Tools:
Applitools: Uses AI to perform visual validation and UI testing. Applitools Eyes can be integrated with Robot Framework to detect visual bugs and ensure UI consistency.
Percy: Provides visual testing and UI review for mobile apps, enabling integration with Robot Framework for comprehensive visual testing.
Visual AI: Tools like SikuliX can also assist with image-based testing by recognizing and interacting with visual elements on the screen.
2. Element Identification and Interaction
AI enhances the accuracy of identifying and interacting with dynamic UI elements, which can be challenging in mobile applications.
Tools:
Appium: Now includes AI capabilities for better UI element identification and automation. Appiumās AI-driven features can recognize and interact with UI elements more reliably.
Test.ai: Provides AI-driven element detection, making it easier to automate testing for applications with dynamic and frequently changing UIs. It can be integrated with Appium for enhanced capabilities.
Functionize: Uses AI to identify and interact with elements dynamically, reducing maintenance efforts for test scripts in Robot Framework.
3. Test Case Generation and Maintenance
AI can automate the generation of test cases based on user behavior and maintain them by adapting to changes in the application.
Tools:
Testim.io: Leverages AI to automate the creation, execution, and maintenance of test cases. Integrates with Robot Framework to provide intelligent test generation and self-healing capabilities.
Mabl: Uses machine learning to create and maintain test scripts automatically. Mablās self-healing tests adjust to changes in the app, reducing maintenance efforts.
ReTest: Utilizes AI to automate the creation of test cases by learning from user interactions and maintaining them by adapting to changes in the application.
