Iāve struggled a little here because Iām not entirely sure what Iām looking for yet. We also have strict limitations on AI usage >_<
Iād love AI to be able to highlight the risks and identify missing requirements, but I didnāt find anything for that. A big thing here is that lately I lack domain knowledge on our product suite, but Iāve not found any threads to pull on there.
Iāve been delving into pipelines because, as much as I have like to avoid the topic (I want it to ājust workā then NEVER need further work), to if we can mitigate some of our challenges.
Launchable:
This was the tools that got me digging further into the topic and I can see how it would be very useful but unfortunately the our tech stack doesnāt seem to be supported.
TestBrain:
The integration is very powerful here. I really like the way that it can create bugs automatically then wonāt fail subsequent commits, which can be handy when weāre working in several large codebases used by many developers around the world. Automatically closing the bug was a little concerning as it might accept an intermittent failure. I may have misunderstood though. I also really liked the idea of the manual testing analysis. That could definitely be interesting for some people in my organisation.
My one concern would be that Iām currently unsure how much is driven through the TestBrain front end. We have a variety of different places to look to understand our pipelines and adding a new dashboard may not be well received. I guess that depends on the audience.
ReportPortal:
The failure reason feature really stood out to me. We are spending too much time having to understand the cause of test failures so using ML to make this easier? Yes please!
Again, the single-entry point wasnāt something overly exciting me because I need more than test results. I struggle to understand how weād use this within our existing pipelines, although if that failure reason feature was seen as worth investing in, perhaps we can explore how we would want the tool to fit into our ways of working.
There are definitely differences in these tools and they arenāt mutually exclusive. I think Iād need to better understand the problems a bit better and then explore tools again.
Hey Testers, here are some AI tools that caught my eye to enhancing testing efforts:
MABL
Overview: MABL is an AI-driven testing platform designed to automate end-to-end testing flows. It uses machine learning algorithms to create and maintain tests, execute them across different browsers and devices, and analyze results.
Capabilities: MABL offers features such as self-healing tests, intelligent test creation, cross-browser testing, and it automatically identifies changes in the application under test and adjusts tests accordingly.
Impact: MABLās AI capabilities can significantly improve testing efficiency by reducing the manual effort required for test creation and maintenance. Its self-healing tests feature helps ensure that tests remain accurate even as the application evolves, saving time and resources.
Testim
Overview: Testim is an AI-based test automation platform that leverages machine learning to accelerate the creation, execution, and maintenance of tests. It offers a codeless approach to test automation, allowing users to create tests using natural language.
Capabilities: Testim provides features like predictive analytics, codeless test automation, and integrations with popular CI/CD tools. Its AI engine analyzes application changes to update tests automatically.
Impact: Testimās codeless test automation and AI-driven capabilities can streamline testing processes, enabling faster test creation and execution. The platformās self-healing tests and predictive analytics help maintain test stability and identify potential issues early, contributing to improved software quality.
ReTest
Overview: ReTest is like having an assistant that helps testers by automatically creating test cases from written requirements. Itās all about understanding what needs to be tested just by reading the project documents.
Capabilities: ReTest reads through project documents and picks out the important stuff to turn into test cases. Itās like having a smart robot that understands human language and knows how to write tests based on what it reads. This means testers donāt have to spend hours writing test cases manually, and the tests are always in sync with the latest project requirements.
Impact: Imagine saving lots of time and effort usually spent on figuring out what tests to run. With ReTest, testers can focus more on actually running tests rather than brainstorming what to test. It also helps keep tests up-to-date as requirements change.
Iāve recently encountered several AI testing tools and some of them are listed below:
TestCraft: This extension stands out as an excellent tool for generating test cases effortlessly. Not only does it provide both positive and negative test cases, but it also furnishes automated scripts for specific scenarios promptly. By leveraging TestCraft, we can significantly reduce the time needed for test case generation while simultaneously enhancing the overall quality of our testing efforts.
mabl: A remarkable web automation testing tool that amalgamates the functionalities of BugHerd and BirdsEatBugs. What sets mabl apart is its intuitive and user-friendly interface.
testsigma: Tailored for individuals with limited coding skills, testsigma excels as a testing tool, ensuring meticulous and precise testing without the need for advanced programming knowledge.
The landscape of AI tools in testing offers abundant opportunities to streamline our processes, delivering high-quality products within tighter deadlines. With these tools, even those without programming expertise can automate their test cases effectively.
We have been using it extensively and itās great but I have found the comparison feature missing from the tool. Comparing 2 executions was one of our key requirements but didnāt work out so well
Also, the auto-analysis didnāt work so well for us. If you were able to solve these issues. I would like to connect and discuss more.
@kato here are some challenges we faced in reportportal
testing need - generate test data using AI tools - select 3
Auto Test Data
Quickly generates test data and allows a user to select from a range of file formats, including JSON, xlsx, CSV, Python and a few more
Simple UI allowing a user to quickly select the data type, property name and options and the number of data rows required
free to use
Mostly AI
Easy to setup and get familiar with the process of dragging and dropping a file to upload and generate synthetic data
Con - slightly confusing to set up and get started to just generate fake data as user has to set up an account and take a tutorial first, compared to the Auto Test Data tool which is more like dive right in and generate some test data yo
Syntho
Seens simple to start generating test data
CON - Demo needs to be booked in advance
Gretel
Mockaroo
Tonic
Here is what i learnt and understood from this task.
Thanks @testingchef this will surely help me in my org.
So here is my answer
Key Tools
Testim: Known for its AI-driven self-healing tests and automatic maintenance, Testim facilitates codeless test creation for non-technical users. It supports UI, API, and performance testing, offering a robust platform for teams with mixed skill levels.
Mabl: Offers AI-powered UI, API, and performance testing. Mabl focuses on self-healing, suggesting test coverage, and integrating with CI/CD pipelines. Itās considered comprehensive for teams with varying levels of coding expertise.
Postman: Primarily focused on API testing, Postman provides AI-driven test creation features through PostBot. It excels in creating tests for individual API requests but may require additional work to build complex test flows.
Report Portal: An advanced tool for analyzing automated test results with AI-driven auto-analysis. It provides insights into test stability and failure reasons, offering a unified dashboard for different types of test results.
Katalon Studio: A flexible platform with AI-based features for generating test scripts and self-healing tests. It supports multiple application types, including web, mobile, and desktop.
TestCraft: This tool allows for test case generation and automated test scripts. It provides both positive and negative test cases, useful for test data management.
Launchable: Offers predictive test selection to optimize test execution. It uses AI to identify which tests to run based on the code changes, aiming to reduce the number of unnecessary tests.
Applitools: Focuses on AI-based visual testing, providing visual comparisons to ensure UI consistency across different versions of an application.
Test.ai: Known for autonomous test generation, Test.ai uses AI to automatically create test cases and adapt to application changes. Observations
Efficiency and Adaptability: Many tools offer AI-driven self-healing features, reducing test maintenance and ensuring tests adapt to UI changes. This leads to higher efficiency and faster test execution.
Integration with CI/CD Pipelines: Most tools support integration with CI/CD systems, promoting seamless test automation within development workflows.
Codeless vs. Full-Code: Tools like Testim and Mabl are designed for non-coders, while others, like Katalon Studio, offer full-code scripting for more advanced users.
Self-Healing: A significant trend in AI testing is the ability to adapt tests to UI changes automatically, reducing manual intervention.
Cost and Accessibility: Pricing varies across tools, with some offering free versions and others requiring paid plans for advanced features. Accessibility and user-friendliness are essential factors for broader adoption.
Test Data Generation: Some tools, like Auto Test Data, focus on generating synthetic test data, which can be useful for performance and load testing.
From this I conclude that the variety of AI-powered testing tools provides options for different needs and contexts. Teams should choose tools based on their testing requirements, skill sets, and integration needs. Tools with AI-driven features tend to streamline test creation, execution, and maintenance, leading to improved testing efficiency and effectiveness.
AI testing tools are revolutionizing software testing by leveraging artificial intelligence and machine learning to enhance test automation, reduce manual effort, and improve accuracy. Tools like Testim and Mabl offer automated test generation and maintenance, while Postman focuses on API testing. TestCraft emphasizes visual testing, and Codeium uses NLP to create test scenarios. Organizations can streamline their testing process, reduce testing time, and enhance software quality with these AI-powered tools.
