Is there any automation tool to test AI applications

Off lately the topic testing AI applications has interested me a lot. I have been exploring a lot on how to test test AI application. The questions i am looking answers for are -

  1. What should be the mindset of a QA to test AI applications
  2. What skills a QA should have
    3.What innovative test approaches can we use

With the complexities these systems offer i believe a lot human cognitivity is required while testing these system. Can you suggest any tool which we can use while testing a AI application. I know the tools would be different for different scenarios. But can we think of any generic tool or may the aresa where we can use automation while testing the AI applications.

1 Like

Naturally, AI communities have developed many techniques for validation of the methods based on AI.

https://digital-library.theiet.org/content/journals/10.1049/ip-vis_19941330

https://ieeexplore.ieee.org/abstract/document/704579/

But, as you said, they are contextual.
If you could detail the problem you are facing, we could drill down on the alternative solutions.

Here are my thoughts on your questions-

Over the past decade, technologies have evolved drastically, there have been so many changes happening in the technology space but one thing constant is human testers’ interaction with them and how we use them for our needs. The same holds true for AI as well. Secondly, to train the AI, we need good input/output combinations (which we call a training dataset). So to work with modern software we need to choose this training dataset carefully as the AI starts learning from this and starts creating relationships based on what we give to it. Also, it is important to monitor how the AI is learning as we give different training datasets. This is going to be vital to how the software is going to be tested as well. We would still need human involvement in training the AI.

Finally, it is important to ensure while working with AI the security, privacy and ethical aspects of the software are not compromised. All these factors contribute to better testability of the software. This is the mindset and expectations of a tester working with AI.

The same QA skills that applies to any application, applies to AI as well. Curiosity, experience, critical thinking, technical and domain kowledge and finally, constant learning are some of the traits that can help us test better.

There are so many AI based testing tools out there, some of them that I personally have used and currently working on are -

Testim.io

It uses machine learning for the authoring, execution and maintenance of automated tests. It focuses on functional testing, end-to-end testing and UI testing. The more tests you run the more smarter the tool becomes to increase the stability of your test suites. It is not a completely code-less tool; you can use JavaScript and HTML to write complex programming logic (if needed) for your applications.

Appvance

Appvance uses AI to generate test results autonomously, and also data-driven regression end-to-end test cases based on actual production behavior. It has advanced validation capability without writing any scripts (validate results). It learns from each test run and makes tests smarter each time. It is a fully AI based system. Works for web based applications including new libraries like ReactJS, Polymer, Angular2, KendoUI, ShadowDOM etc.

Functionize
Functionize uses machine learning for functional testing. It is very similar to other tools in the market in terms of its capabilities

I am able to provide only 2 links, so could not link the other tools and provide more links to other AI based testing resources. If you want on info on that, just contact me