I scrolled through the TestMu conference yesterday, and so many of the talks were dedicated to AI. I understand that there is much to learn, and I’m also worried that we might lose the nuance of testing by being so focused on AI. Does anyone else feel like this? Are there conversations you’d like to have around testing that don’t involve AI?
14 Likes
I’m hearin’ ya, Judy.
I’d love for us to have more conversations about exploratory testing, such as chartering, note-taking, time-boxing, and debriefing. That kinda thing!
So much to learn and value to be realised with practical approaches to exploratory testing.
When it comes to “quality is caring”, such approaches are a game changer.
1 Like
I do see AI being present everywhere - so many tools being introduced based on AI promising to reduce human efforts and time. But I feel while we use AI, we loose our thinking capability, creativity and eagerness to do it uniquely.
I feel AI can’t be used independently for anything E2E without having base concepts clear, without having basic knowledge about the area we are looking help from AI.
Testing areas like integration, communication between multiple modules, session management among multiple devices etc where we need to keep an eye on the services, 3rd party integrity, DB etc human eye and expertise are required. You can not explain your system to AI nor AI can see all the connections but you can see what’s breaking, what’s happening in all the areas of the application
3 Likes
It actually amazes me how many AI talks there are, there is always a lot of feedback about “too much AI” and STILL they keep putting more AI on the program. At Brewtcon in Belgium, they specifically didn’t pick any AI topics 
because people are sick of it 
10 Likes
Actually, I’m avoiding meetups and conferences because of the same reason. And I’m not even attending Testu , not saying all talks are bad, but somehow at the end of the day, everything ends at LLM, MCP.
I’m saturated with this kind of content, and that’s why I avoid even putting such content on LinkedIn, as there are already many experts who put some fascinating theories related to AI, LLM, etc., every day.
Somewhere there is lot of noise that if you don’t know about AI then you are outdated and may be that’s also one reason everyone s talking just about same to gain attention.
6 Likes
It’s probably going to sound ageist, but I work in the embedded and industrial sector mostly, and for me, safety and predictable risk has been key for over 30 years. AI was actually there on day 1 though all along, so I’m finding the problem with AI is that it’s a “rent seeker” at the moment. It’s renting space in our power grids and our brains for free. t’s only paying it’s way in limited number of applications still. 30 years ago, it was in things like predictive maintenance, AI diagnostic tools, and of course, there was AI generated or code-generated music back then. AI has become a land-grab. But even though to me it has limited industry application, it’s an increasingly useful tool. Only trouble is that the technology stacks seem to lack “stickiness”, so it’s still in the grand scheme of things , still very cost effective to keep changing providers as a consumer.
So yes, we are learning, but not about portable AGI that sticks in my world of high reliability software and hardware at all. I mean our teams use AI and “large data” tools to build circuits, but that’s not what users see at all, they just see a machine that works correctly. It has uses, but not in my QA testing at all yet. I get more mileage out of my deck of Test Sphere cards. And I’m having much more fun learning advanced Python programming than any AI stack can provide in a job where, frankly, AI has not got an experience dataset for my job yet. Probably because my actual target market is still undocumented and always will be.
5 Likes
The more focus there is on AI, the more the community needs to look at our thought processes. The longer I’m involved in software testing, the more the value of thinking about testing seems to be highlighted. Skills cannot be applied without thinking first. AI doesn’t think, it doesn’t innovate, it has no intuition or empathy. So we have to think even more when we use it.
5 Likes
The problem with so many AI talks is that many are not actually helpful because people don’t really have direct experience.
Our focus at TestBash this year is on Quality Engineering, there are a couple of AI related sessions for good measure, but the overall theme that I feel is important for us is more ‘holistic’.
10 Likes
AI is just a new religion. Or you believe in it, or not. Or maybe you’re just agnostic.
And depending how much time you spend preaching and praying, you still have time left in you daily day.
1 Like
So true. I still can get over the absolute confidence people put in eg. Chatgpt for their most important projects. Currently it can’t even draw a map of Germany with it’s federal countries. I have gotten good results out of it, but rarely in specific questions. I used a testing tool with built in LLM that claimed to just need the requirement and would then put out the corresponding test cases. It did but only after three runs plus many unnecessary ones.
To me, AI is like many predecessors in technology- It will replace “dumb” work but won’t be able to “invent” stuff. Look at the history of the Spinning Jenny - it put many weavers out of work but never eliminated the need for quality control.
2 Likes
If you took out the stuff that is trying to sell AI products it may be a bit better.
AI for me is an area worth a level of focus on and its worth getting hands on experimenting with it so you can maybe see new ideas that nobody else is talking about yet. I want to see more discussions on AI in exploratory co-testing approaches, real examples where it can be used for running experiments and investigative testing alongside some of the downsides of going down that path.
There are a lot of experiments that need doing and likely need a lot more testers doing them to see if value in approaches. For example I like to see some experiments where testers have found issues during exploratory sessions and then looking to see if they could have been found using prompts at code level, and strengths and weakness of both approaches.
I’ve seen before a load of people skipping learning testing and jumping straight into automation and having good careers oblivious that they could be adding more value. Some of the tools are now bypassing the idea of learning automation and going straight to llm generated automation. There are likely a lot of pros and cons on this but there is also a whole load of bias.
We may end up losing a lot of that deeper thinking about testing as a whole and shifting more towards shallower testing, I see some signs of this already fast shallow testing and the quality bar may drop.
I used this quote the other day, “I like the model of testing that on a journey to mars they will happily blow up a spaceship and call it a successful test because they learned” whether AI will take us even closer to the manufacturing model remains to be seen.
So my thought are going in many directions on this, experiment with AI I am all for but keep a skeptical mind.
3 Likes
I think there is less to learn about AI than you imagine, and I spend no time on it at all. Sure there are lots of experts talking about it, some of whom have been using AI for as much as a couple of weeks, but I am unimpressed with everything I’ve seen.
I will pay more attention when Bach and Bolton say it’s ready to be useful, at which time it might take a day or two to catch up. I have no fear of being left behind because most of what’s being done at the moment is nonsense.
It’s worth keeping an eye on what the more sceptical developers say about it, especially the security researchers. Most seem to be pretty horrified at how bad AI coding is. There is no reason to expect AI to perform any better when used for testing.
I have always been scathing about the low level of testing skill in our profession - it’s extraordinarily poor for the most part. Testers should spend many years (at least five, perhaps ten) learning to do testing to a high level of proficiency before even thinking about using automation, let alone AI, otherwise they are just like a monkey with a machine gun.
2 Likes
It does feel a bit like some people are too much in a hurry to “outsource” thought work. As Ady said, AI can’t think, so it’s still important that we can think critically. I think it’s also important to use AI as what it is - a tool for people to use, not some accountable being you can grant responsibility to and trust to make good decisions without oversight. After all, most of the AI a lot of us use is just an LLM. They’re about predicting language, not understanding the meaning behind it. And ultimately, they follow what other people say, but so much of what we do, as testing specialists, is speak against what others are saying, to highlight issues and risks.
3 Likes
I signed up to TestMu but I have to be honest, it seemed to be more about quantity than quality. As you say AI was in ya face, on 90% of the talks. I even saw a talk on DORA metrics that suddenly turned into how AI can make the world better. The conference appears to be selling you how much you need AI to then switch you to, “oh hey, we have AI tools you could buy”. I am very much turned off from TestMu.
To me AI, is just another tool with a different perspective. Use it where it gives you benefit, don’t where it doesn’t, but you will have to learn those boundaries for yourself. You remain responsible for your work, AI doesn’t. Not sexy enough for a conference talk I know 
4 Likes
In my work environment AI is a continous discussion: some team members hate it, other team members make use of it… AI can’t be denied any longer. I use regulary AI to be informed, to get to solutions more quickly, but I keep my eyes open. For especially test purposes, I believe that AI can be involved where you (the human) knows what (and how) is asked from AI to help in order to be more efficient. The same communications errors I see between people and teams (especially off-shore). You can have the same confusion with AI if the question/task is not specific enough. I would say: keep focus on the opportunities AI offers but focus in the mean time on how to use AI efficiently and wise.
“AI is revolutionizing testing.”
Every second post says the same on LinkedIn. Seems like people are seeing more revolution in testing through AI than AI is actually doing.
Everyone is like - test cases are dead, test planning is dead, automation scripting is dead, functional testing is dead, etc… all the fundamentals of testing are dead… and what is alive just testing by AI, and guess what same people will post “after using AI, brain is dead, and so is creativity. “
That’s how conversations are these days… a similar pattern where less talk is about actually improving the testing and more talk is about improving the testing using tools. But tools are just an aid to the testing, and testing is beyond tools and AI.
2 Likes
I can indeed see the point. There has been a lot of discussion of AI in the recent times, especially in certain tech communities or in conferences such as TestMu. The topic of AI brings much potential to the field of testing, but it may overwhelm the classic fundamentals which make testing worthwhile in the first place.
When I do testing, I do not conceive of it as automation or ML models, rather it is really about knowing the application, predicting edge cases, and assuring quality from the human side. There is a nuance of testing, such as critical thinking, creativity, and intuition, that an AI cannot really in its embryonic stage replicate.
I want more talk for provisions apart from AI on testing itself; like how to improve communication between developers and testers, creating and reviewing test strategies, and knowing manual test activities that still can bring value these days.
Is there anyone out there who thinks it might be wise to discuss a little about AI and also some good ol’ testing fundamentals?
As someone who’s continuously striving to learn testing and automation to fit the job market’s standards, I noticed the importance of being guided by people who know better than spouting AI noise.
For example if I hadn’t learned the basics and the concepts of testing and coding in Java, it would be impossible for me to understand if an AI’s coding suggestion is correct or not. I might get easily swayed by AI codes scripting the test cases in its own way and not my way. Because of this I have remained skeptical of any suggestions unless it’s proven to be of good use.
Right now AI is just a tool, and as a Jr level tester I’m actually wary of using it.
Good thing the more senior, experienced testers I know are the people who agree that there’s more to learning than using AI 
3 Likes
Hear hear.
I feel everyone in this thread, you are not alone 
Actually, whatever platform you open - there is AI in any form or shape at times it overwhelms and might drag people especially newbies in software testing industry into ‘imposter syndrome‘ because it projects lots to digest and gives perspective we are lacking latest trends and everyone else has it ‘figured out’. Obviously which is not the case.
On the other hands we also hear: “You wont lose your job due to AI but to someone who know how to use it“.
That’s my firm believe that AI can never replace human neural thinking, holistic & heuristics. As long as end users are human, we need humans to plan, build, test & monitor for sure. (AI even made us think about it twice)
We are here to stay, lets take over AI before it takes over us (pun intended ).
We as human need to believe that we are not replaceable as simple as that! However for sure we have to work towards it. 
Edit: Would reshare my poem: Trip to the Memory Lane - A Poem | Ministry of Testing
3 Likes
Just shared a meme: 404 Earthly Feeling Not Found | Ministry of Testing
3 Likes