lately, I have published a video about AI features for software testers on my channel:
In the video, I am presenting current AI features software testers can already use. I was wondering if you use any of the mention tools in your environment, and what are your experiences with them? Also, I would be interested if you use already other AI tools or features on a daily basis.
I am really curious to see your comments on this topic, since AI is (from my point of view) a topic that is disrupting many industries similar to the smartphones have done from 2007 onwards.
Hi Daniel, thanks for the video. Itās clear that AI tools will significantly impact the industry, though I believe their influence extends beyond just testing. The entire development and delivery process, including testing, will be transformed. Your smartphone analogy is apt. It reminds me of how chess engines revolutionized playersā tournament preparations. AI tools seem poised to be a similar game-changer for our industry.
Moreover, AI tools themselves require rigorous testing, which opens up a vast and intriguing field of study. Far from becoming mundane, testing is evolving into an even more dynamic discipline.
Currently, many tools are available to streamline routine tasks and enhance daily activities:
Copilot is outstanding (especially with the new Copilot Chat feature) ā itās incredibly useful for code generation, migration, and other common tasks. Intelij IDEA AI assistant is also helpful tool.
AI-powered locators and test data generation are promising areas. Iām keen to explore tools like DevTools AI, although I currently donāt have a UI to test in my role, thus posponed the investigation.
There are many other potential applications. Iām curious about tools that assist with requirements analysis, test documentation, and test design. If these arenāt available yet, Iām confident they will be soon.
I have tried putting graphQL query definitions directly into Chat GPT 4 and prompting for test scenarios and automated checks based on this. The results were a bit mixed but better than I expected.
I have also found the Postbot feature for Postman quite useful.
You are right. AI will change the whole industry. I really hope that companies will not do the same mistakes, when they tried to replace testers with automationā¦
I am really looking forward to all the AI stuff, itās a bright future for testers, checking what the AI has done, to see if itās true or just wrong.
Now is the perfect timing to get into this topic, to learn all about LLMs and the tools to be ready for the change :).
I might be unique on this but I feel like AI is almost always misused in a testing context. This is because plugging it into a use case requires a certain degree of fault tolerance and in general testing is about being fault intolerant.
The two exceptions where I thought āwow this really works wellā were:
End to end testing LLM applications. This means throwing a creative (i.e. high temp) LLM at our LLM customer service app and telling it to be nasty, sarcastic, snide, etc.
A machine learning app by Mozilla for deciding the order of the ~48 hours worth of end to end tests based upon the change to get the MTTFF (mean time to first failure) down.
I agree. Many companies doing the same mistake again. Remember the days, when they thought test automation can replace testing :D? Well, we all know that this is not correct. The same thing is now happening with testing!
I think that with the raise of AI, there is more work to do for testers than ever before. The results of an AI need verification. Testers need to get into the algorithms of LLMs.
I think the role of testing will change in the next years. We will use AI to test better and we need to have tools to test AI. Letās start with our ās
100% Rosie! The sad but is that many companies donāt think that way. I see it on a daily basis. For them, itās just about saving money in the first place. And if someone is telling them,
āHey buy our fancy AI tool, and you can replace āresourcesā!ā
They do believe them. To be fair not all companies think that way
Many of them still need to learn the hard way that testing is so much more than clicking and tapping on a productā¦
I discovered Postbot from Postman at their AgileTD booth from Postman. Looks awesome, but has some GDPR related issues which I want to investigate further. Am in contact with Postman about this since the AgileTD.
Hi Daniel, thank you for the interesting list. I am glad to see that many test automation tools are using AI. I would also like to mention Tailblu as an AI-powered test automation tool with following features:
,
ās 
