I wonder in which activities could ChatGPT help you in your testing?

Hi all,

With the latest openAI trendy about “ChatGPT”, I’m curious to know your opinions about it, do you think it could improve your testing tasks?

In my opinions, it could be effective in testdata generation, in devops pipeline technical tasks, maybe giving new testing ideas that could be added to exploratory testing…

I also have more questions for you,

  • How can we become more productive using its hacks and (sometime smart) replays ?
  • What are the new tasks that could be easily automated with chatGPT ?
  • What new challenges we could encounter in the future as testers ?
  • Shall we also include them into our testing strategy?
3 Likes

I’ve used it to read the long article for me and provide me a short summary, it’s fairly accurate but a little too literal sometimes, but still it seems useful. Maybe the next thing to try is to have it generate some test data!

2 Likes

It can struggle with test data. Also math. And it does it with authoritative confidence as well. Check out some of these sweet birthdates.

4 Likes

that’s shocking :face_with_monocle: ! 45 day or month of birth :joy:

1 Like

Looks like the sort of progression I was using to create test datasets 25 years ago when I was working in utilities regulation - though those were for numbers like “Length of water mains refurbished” and “Number of properties connected to mains”, and the progression was over time. And I knew what the likely upper and lower limits of each data point were.

This got interesting where one client started falsifying their data returns and were providing numbers with similar progressions to my test dataset as their live data. As real-world pricing decisions were based on this data return, there were consequences (a £35 million fine and an investigation by the Serious Fraud Office).

Which suggests a use for an AI system, albeit more in validation than in testing: using ChatGPT output as a benchmark against which future data submissions might be compared. If the progression patterns within the submitted data was too close a match to a ChatGPT “benchmark” dataset, the system would raise a flag. Especially useful if the dataset has something like 40,000 data points, like the one I was working on.

2 Likes

To be fair, it’s not October 2025 yet. We could get a 55th of the month by then. I’m not willing to say it’s impossible.

3 Likes

I did not use it for that. I will give it a try.

3 Likes

that’s a good point ! talking about blogs I’m also thinking to make a serious interview with chatGPT and then comment about the result in a funny way :sweat_smile:
BTW, chatGPT has funny jokes to tell :joy:


a bonus for @mirza

I love those explanations! :laughing:

OMG, what a great idea! I’m going to steal it :sweat_smile:

3 Likes

I used it to help me get started with the code for API tests, it worked pretty well actually. Not sure how it would work for complex scenarios, though.

1 Like

@vincentferreira recently shared the following treasure trove of handy ChatGPT prompts:

Nice one, Vincent. :smile:

3 Likes

I asked it to generate some tests for a date picker, basically test for input ranges, and I think it did a pretty decent job out it .

2 Likes

In terms of code, it’s been working pretty great for things thats more commonly known. I used it to help me write k6 tests for an AWS redshift cluster.

It gets a bit complicated when there isn’t a lot of resources about a given topic. My recent experience left me with library functions that doesn’t exist :see_no_evil:

7/10 in my opinion overall :+1:t2::raised_hands:t2:

3 Likes

Some areas where I have tried it:

1 Like

A reminder : do not use chatgpt to analyse customer sensitive data in any way, or any feature work that is not public. it will leak that work for you. Even grammar checkers will leak your little snippets of documentation you wrote and just wanted help correcting typos in, back into the web via it’s learning set. Any code it writes may well see you getting into trouble for stealing code (assuming the code works), because of the same learning set leaking behavior.

That said, it’s a great tool for bouncing ideas of of. And great for researching , often better than google.

2 Likes

I love ChatGPT for a lot of reasons:

  • Writing emails

  • Writing invitations

  • Writing documentation

  • It’s actually good to give several examples to explain certain things in the testing world.

  • It’s good for remembering frameworks, what I mean by this is let’s say I haven’t used JMeter in a year or 2 and I need to start using it again.
    And you ask the question to ChatPGT “How do I make a performance testing script in JMeter?
    It comes up with a very good direction and explanation of how to do it.

  • You can paste an API request in it and ask how to test it. You can keep asking ‘is there anything else you could test’ and it will provide you more

  • Also this is pretty nice! “Find bugs in the code”
    ChatGPT Inside VSCode. Your new coding partner - YouTube

3 Likes