I see GPT-4 has started rolling out to users who sign up for ChatGPT Plus, And there’s an API waitlist too.
Details here and a technical report about its capabilities.
I see GPT-4 has started rolling out to users who sign up for ChatGPT Plus, And there’s an API waitlist too.
Details here and a technical report about its capabilities.
From a philosophical perspective, a tester does not/should not look for ready-made answers. A tester explores, is heuristic-driven, and is context-sensitive. To that end, chatGPT will be yet another tool in my toolbox to get some tips (not exactly answers) when I need some.
chatGPT cannot be the Test Oracle, if people are looking for one.
It’ll be interesting to see if/how this has changed in GPT4.
It making things up but stating them as answers without qualification is perhaps not exactly the same as an overconfidence bias in people. However, I wonder if there’s something about how we intuitively get a sense we’re encountering this bias in human conversations, but with ChatGPT we don’t get the same cues that could be looked at. Maybe it could state a confidence level or be asked to cite its evidence or sources… Maybe it can do this now… Time to go playing with it ![]()
I’ve just posted on linkedin about what happened when my research questions got put in ChatGPT by my academic supervisor… Unsurprisingly it couldn’t answer, surprisingly it didn’t try to answer… more on the blog post here: Isabel Evans on LinkedIn: Research: slow but sure, and an encounter with ChapGPT
From my experimentation, it feels like ChatGPT can help someone be more productive but will not replace anyone soon. I have used it to help create some simple Selenium scripts, but I have a programming background. I do not think it will turn a non-programmer into a Selenium developer. It can help with the skeleton of a test plan, but needs your experience to fill in the details.
It is happening right now @simon_tomes . Here is how AI builds and executes full end-to-end test: https://youtu.be/rZEmxEmwFvs and here how you can mix and match AI with regular code and take advantage of improved test stability: https://youtu.be/8-LcECT4jjQ
The more I play around with ChatGPT - the more impressed/scared I am of it.
We are behind schedule with test design based on the user stories that are ‘ready’ - and I decided to try out ChatGPT in designing test cases based on the content of the story - The results were so impressive - It even created in Cucumber when requested
Obviously, I will need to review what its populated to make sure it is correct, but doing this will save considerable time for us ![]()
I am scared of the quality of the current & future software.
The quality is already at the worst it’s been in history.
Software is generally made to solve problems or make things easier for humans.
The more we move away from the human aspect when testing software of interacting, exploring, and experimenting with software - the less the software will be made for humans.
In my normal outside-of-work life, I encounter about 30 bugs each month, in about 50 different applications, that waste my time, annoy me, or block me from doing what I need to do.
I used to be pissed off each time, swear, close the app, go for a walk, or delete the app.
Recently I just avoid as much as possible using software, unless it’s no other way.
If you read the articles linked inside the post you can feel how I feel:
Software testers, in a decaying software building process, don’t try to push for things to get better, instead, they participate in making software worse, with all the ‘bloatware’ software practices they add to their work.
I ran some experiments to see how useful ChatGPT would be to me in my testing role.
Writing a test approach
Writing exploratory test ideas
Teaching me stuff
What I found is that for my areas of testing, ChatGPT is as useful as a Google search, which means Garbage in / Garbage out. It’s only as useful as the content that there’s out there on the web.
Loads of people think Exploratory Testing is an ad hoc bug bash only and write about that? Thats the approach you get!
So for me, It’s good for the basics… but what comes out of it needs a lot of verification and rework by an expert.