đŸ€– Day 6: Explore and share insights on AI testing tools

OK, I was able to play around with Testing Taxi a bit. It makes useful suggestions. As far as helping me test, the feature to copy/paste snippets into input fields is nice. However, it doesn’t add a lot more value to me than the BugMagnet extension does, and BugMagnet makes no claim to using AI. It’s hard to see how this tool is using AI.

3 Likes

And, testCopilot, aka Checkie.ai, I just can’t get it to work at all. Just now it gave me an “undefined” error. It seems to be trying to scan the web page for various things such as security and usability, but, even when something does come back, I can’t figure out what to do with it.

1 Like

Thanks, I’m going to try this one next. Do you think it’s more useful than just using ChatGPT? (love your mind map, too)

2 Likes

Hmnm, I can’t get this extension to do anything. Is there a user guide anywhere? I looked at the details in the web store and couldn’t find anything. It pops up this window, but nothing responds when I click.

1 Like

There is a bug in the extension if you use dual monitors. I discovered it today.

Try using it on your laptop monitor only?

Also, please check my demonstration video (link in my original post).

2 Likes

Yeah, this is a lot better than raw GPT prompting.

Thanks for the feedback on mindmap.

1 Like

I was on my laptop, but I’ll take a look at your video, thanks!

1 Like

This looks very interesting to me, I want to try it as well. I just registered with google forms they offer. @lisacrispin do you know how much time it takes for them to review my registration and grant me access to it?

1 Like

Hi,

I was looked for AI generated tools for visual testing. What I found:

:small_blue_diamond: Diffblue is AI automated tool that uses machine learning to create tests. It can analyze codes and generate tests. Also, it can identify potential issues that can affect any user’s experience.

:small_blue_diamond: Testim is another UI/UX testing tool that can identify critical elements in any web application interface. It generates tests that analyze all user’s interactions and their behavior. Testim is usefull tool for testing the user’s experience of any web application. Meanwhile, it shows potential issues like slow loading times, etc.

:small_blue_diamond: Applitools is an AI tool that uses machine learning to detect visual problems in mobile and web applications. It’s compatible with modern web applications and can detect visual issues. Also, the tool uses AI algorithms to analyze visual elements and compare them for results. The tool’s algorithm also identifies color, font, and layout issues. It can also detect errors related to missing buttons and bugs.

:small_blue_diamond:Katalon Studios is a UI/UX testing tool that identifies the elements of a web application interface and generates tests to cover user interactions. It also has an in-built spy feature that allows developers to check the web elements and generate automated tests. Besides, it allows developers to test specific UI elements quickly.

2 Likes

Oh my, today’s prompt. Took me quite a while to to balance my attitudes:

  • There’s no chance I would spend time exploring yet another commercial tool
  • There are very few AI tools that are anything more than automation

So I invested an hour into playing with TESTAR. It’s a multi-year multi-research-project created BSD-3 Clause licensed tool with a publicly reviewable repo. GitHub - TESTARtool/TESTAR_dev: TESTAR, automated testing through the Graphical User Interface from research lead by Tanja Vos.

The promise is quite hefty at https://testar.org:

shifts the paradigm of GUI testing: from developing scripts to developing intelligent AI-enabled agents

I could imagine this is AI for the overall goal of acting humanly, but the focus is on robotics (manipulating objects and moving about).

It uses windows accessibility Assistant and Selenium Web Driver to randomly click around in configurable amount and length of sequences. It has a few preprogrammed oracles they call implicit that are very explicitly looking for crash, hang, suspicious titles.

I run it against multiple windows UIs with the product I am testing and can’t confirm yet the claim that it “beats manual testing” in results, but if manual testing was clicking all buttons looking for crashes, we beat that already years ago.

While I am writing this, Testar is still doing 50 sequences with 50 steps against the Windows UI I most recently have been playing with. I will report back if it finds anything.

8 Likes

Hi Everyone!
I’m enormously filled with gratitude for your posting of tools analysis yesterday and today!
Being completely new in using to AI, I discovered a virgin field for experimentation :muscle:
and will devote to it the next several months and not one or two evening as this tasks requires :blush:

dear @testingchef, this task is time-consuming for any seasoned tester, but for a new one unbearable to do it during 1 day, hope this humble feedback will give you desire to revisit your task for future candidates :four_leaf_clover:

1 Like

50 sequence 50 steps concluded “No Problem Detected” but I actually detected two problems. The story of that went on my blog: A Seasoned Tester's Crystal Ball: A sample of attended testing

4 Likes

It took a couple weeks when I asked, but that was last fall. Good luck!

1 Like

Test.ai may be helpful for creating test cases:
quote
ML-based element detection finds elements visually - just like a human. The Interactive Test Composer allows you to control a running AI-Bot as it navigates your application, collect and label data, and build tests in real-time. Simple drag and drop codeless interface.
unquote

1 Like

Hello,
I was looking for a no-code tool to test a hybrid mobile app on local devices. Suddenly there is not much choice - UIPath, Catalon, Testim and Repeato. The simplest would be Repeato, the most complex would be UIPath - that is, start creating a test.
If you know anything else, please share.

3 Likes

I will take my time and try some tools that some of you already mentioned.
Today the time was not on my side and I think this task requires some more time to be able to review different products.

My candidates are:

  • TestCraft - help with test ideas are always welcomed and I see it has also the possibility to generate some scripts (Playwright and Cypress) and that is cool!
  • PlaywrightGPT - I want to take this one for a spin - really curious about it
  • Testim - UI testing with AI, and they have free community plan, I will try it out as well
  • Testing Taxi - also looks interesting as the way to assist with testing
4 Likes

Why choose Katalon Studio

Because the slogan and introductions from other community members made me want to give it a try:

  • AI-powered authoring

Generate test scripts instantly. Explain code with 1 click.

  • Self-healing

  • Write better tests faster, with no-code or full-code.

Easy for beginners, yet powerful for pros.

Flexibility to test any app.

Download link

https://katalon.com/download-next-steps

A simple attempt

I conducted test case recording and debugging through a simple business scenario and tested the self-healing script functionality after debugging the test cases, which basically worked well.

The self-healing feature can save some time that would otherwise be spent on debugging and fixing scripts.

I have not used the AI-powered authoring feature, and I will provide a conclusion after trying it out later.

Answering Questions

Tool Overview

Katalon Studio helps teams write better tests faster with easy, flexible, and AI-powered solutions.

  • Ease of Use Easy for beginners, powerful for pros. No-code or full-code options.

  • :magic_wand: Test Flexibility Test any app, integrate with web, mobile, API, desktop, and more.

  • :zap::heart: AI Integration Boost productivity with AI-powered testing and integrations.

  • Self-healing

Key capabilities

  • AI Integration

  • Easy for beginners

  • Self-healing

  • Support web, mobile, API, desktop

Your perspective on their potential impact on efficiency or testing processes

I personally think the following points can impact efficiency or the testing process:

  • Generating automated test scripts through AI suggestion words can improve the test script writing efficiency and also reduce the reliance on QA with high coding skills to write test scripts.

  • The self-healing ability of AI can quickly fix scripts that run into errors, which enhances the efficiency of both test case creation and regression testing.

3 Likes

Hello Lisa I actually did a video on it this week :smiley:

So I can try and help if you need

1 Like

I can’t figure how to even download the thing. Am I missing something?

Testing Taxi is not that good.

3 Likes