Bug Reporting & AI: Helpful Tool or Risk for Testers?

Hi testing community,

I have a question about bug reporting and AI, and I’d really appreciate your thoughts.

Is it a bad thing if I describe a bug to an AI tool (like ChatGPT), let it write the bug report and then send that report to developers?

I’m asking because I sometimes worry:

Does this affect my ability to think better or write clearer bug reports on my own?

Am I really practicing the skill if I use AI as a helper?

I work as a Junior Software Tester with one year of experience, and I’m currently the only tester in my company. Since day one I haven’t had a team lead or a senior tester to guide me so sometimes it’s hard to know what’s acceptable and what’s not when using AI at work.

To be clear, I don’t just copy and paste. I always read, review, and understand what the AI writes before sending anything.

So my real questions are:

Can relying on AI for bug reports be harmful in the long run, especially for juniors?

Is it a bad habit if a new tester depends on AI for writing bug reports?

Where should we draw the line?

Should I stop doing this completely since I’m still a junior?

I’d truly appreciate your experiences and advice.

Thanks a lot for taking the time to read and share your thoughts :heart_with_ribbon:

1 Like

Hi @Nim ! Thank you for writing this out. I hear you on being the only tester in your company. I am in the same boat as far as being the only QA, so happy to support you how I can.

I took some time to read your questions and meditate on them. I think there are a few things to consider.

First, let’s step away from language that implies good or bad. You aren’t doing anything wrong, so taking a more neutral tone may help relieve some of the stress and the worry I hear in your writing. Let’s start from a more neutral place. (And, for sure, I have my worries about AI, but in your circumstance, I think you are trying to do the best you can. I think all of us are)

AI is a tool, like anything else. Some people will sing its praises to the rooftops (probably because they are peddling it to the market) while others will take a doomsday approach and proclaim that it’s going to be the ruin of us all. I try to find and listen to people who are more middle-of-the-road and realistic. AI can be helpful, sometimes. AI can be hurtful, sometimes. Identifying where and when it veers off into these very different directions requires we pay close attention.

For me (and I’m 10 years into the game), I use AI where it makes sense. If I am building knowledge about a project, I don’t give it to AI straight away to summarize. As I write test cases and work with the new feature, keeping AI out of the process is helpful because as I document what I should expect the feature to do, the feature gets built out in my head (if that makes sense) and makes it easier to recall when I’m testing it or working with it later. I also have the memory of Dory the fish, so remembering how things should work is crucial for me. I don’t use AI to build out bug reports because the majority of the time, I know how something should work, and I’m fine creating the report myself.

For clarity, would you be willing to post an example bug report? I’m curious to see what a collaboration with AI in this case looks like.

As a junior, and working with AI, I have some additional questions:

  • What benefits do you feel like you are getting from using AI?
  • Have you asked the developers if your reports are helpful? (Getting feedback from the developers is crucial here since they are the ones who are the recipients)
  • What if, instead of using AI to write the bug report, you reached out to a developer to get their feedback?

For me, I keep AI out of the loop if I want to learn about the feature I’m working on and need to regurgitate that information later on. Is using AI helping your learning? Are you able to recall what you need to when asked about it at a later date?

The other thing that I struggle with is that because I am the only QA, I struggle reaching out to developers (aka, real humans) for feedback on what I’m doing. It’s scary pairing with them, and I work with some really great people. Using AI as a go-between for information might prevent you from:

  • Learning what else the functionality should be doing. AI won’t be working on it. The developers will be.
  • Fostering relationships with the people you work with every day. As you make yourself vulnerable to your team and reach out for help, you will make connections and will learn to be braver.

As a QA, we have to be the ones bringing up all kinds of conversations. Being willing to get feedback from our teammates is crucial.

I’m curious if this week, instead of using AI to help create the bug report, you reached out to your dev team. Let them know you are working on an experiment, and if they could provide feedback for your report, see what kinds of conversations you could have as a team, versus siloed out to ChatGPT, where no one can view those conversations later.

Ah, yeah. That is another point. Having conversations out in the open instead of DMs or AI convos creates unnecessary siloes. The more info that’s public and available to your teammates, the better.

You shouldn’t need to ask devs for feedback on every bug report. But once you get into the swing of things and get solid feedback from your teammates, your confidence will increase. With or without AI :wink:

Hope this helps. Feel free to ask me any clarifying questions. I’m happy to help however I can.

4 Likes

This is 200% of what I needed to read today.

2 Likes

Good question. I go with @jmosley5 , communicate, with people. Not having a mentor or team lead is your struggle I suspect, and I would actually carry on using AI tools, if it actually improves your confidence at filling in the bug-reports with enough information.

But I am surprised it’s possible to use AI efficiently to write test reports or bug reports, that feels like overkill. But who am I to judge. I would be having conversations about even more productive uses of AI, because you are clearly very capable already and are looking to do better. Thaty will keep you in the drivers seat.

2 Likes

@jmosley5 Hi Judy, thanks a lot for your support, I truly appreciate it.

Also I’d really love to learn from someone with your level of expertise so many thanks for sharing your thoughts.
Regarding the bug report examples, I will definitely attach one below this reply.
As a junior, the main benefit I get from AI is mostly in writing bug reports.

English isn’t my first language tbh, I’m not bad at it, but it can take me some time to write a clear report

I understand the expected results and everything, but AI helps me phrase the issue better, write it in English faster than I can, and organize the idea in my head, Since work is always rushed, I try to write each issue quickly and clearly so it won’t be rejected or delayed because something wasn’t understood

I’ve already asked for feedback from some developers and they said that it was okay, they can understand the reports, and if anything isn’t clear they check the attachments I add.

One senior backend developer also told me he doesn’t find it difficult to understand my reports, and if anything is unclear we can just have a quick meeting to clarify it.

About your advice to write reports myself: recently I’ve actually been trying to do that, because I worry about what I mentioned before that I might struggle in the future to write bug reports clearly without AI. So right now when things aren’t too pressured I try to write them myself to get used to it and I’ll get new feedback from the developers once I write the bug reports myself.. My only concern is whether relying on AI for too long could affect my performance later.

  • Can I ask- if you remember- how long it used to take you to write one bug report and one test case at the beginning of your career?

Again, thank you, from the bottom of my heart, for your effort and for sharing your experience with me. I truly appreciate it​:heart_hands:

1 Like

@jmosley5 This is an example of one of the bugs I reported a while ago. I described the issue along with the expected and actual results, and then AI wrote the detailed info

Title: API allows Reviewer role to approve Publishing Request although approval should be restricted to authorized role only

Steps to Reproduce

  1. Login using Reviewer account and obtain a valid token.

  2. Open Postman.

  3. Call the Approve endpoint for the Book Publishing Request.

  4. Provide required request body

  5. Click Send.

Expected Result

  • Reviewer role must not be able to approve the request

  • API should return: 401 Unauthorized and Response message should indicate lack of permission ( “You are not authorized to approve this request”).

  • Approval should be allowed only for authorized roles (Department Manager) according to the workflow.

Actual Result

  • API returns 200 OK / Success.

  • The request status changes to Approved

  • Approval action is recorded in request logs although it was performed by Reviewer.

1 Like

Hi Conrad,

Thank you so much for sharing this and for your kind words of encouragement, I think you described my situation really well.

The thing is I’m not letting AI “do the work” for me. It’s like I describe the issue exactly as it happens to AI and also describe what should happen instead, then I write it in our bug tracking tool.

I feel like this makes my reports clearer and a bit faster maybe, at least that’s how it seems to me. But I’m definitely willing to change it if it’s considered a bad habit, or if it could negatively affect my skills and professional growth

And that’s actually what my question was about in the first place, if this could affect me in a negative way, then I’ll try to stop doing it

Many thanks for your help i truly appreciate it

1 Like

I definitely struggle to write - but when i write a bug ticket or a test report I’m in a different mode - my bollocks-detector is enabled. It’s technical writing. and AI, once you tell it what you want is damn good at setting the “voice” and stopping my write garbage. This is an excellent idea.

I also write recreationally, I have been using that as a way to learn to write “better” , pardon my gremmer, and get my idea across faster. It’s hard if your brain is inhabited by squirrels and means it takes an aGE. So yes I might be keen on a non-invasive writing assistant, but at the moment I’m still struggling to get my phone to recognise which address I want to navigate to, so I feel blocked by “AI” voice assistants for now at least.

Thank you for clarifying! A couple of reflections come to mind for me.

You mentioned that English is not your first language. I can related in the fact that when I started in QA, speaking technically was not my first language :sweat_smile: I was a stay-at-home mom 10 years prior, so translating the issues I saw took some time. The best thing that helped me (and I think it might help you too) is trusting your teammates. Trusting that they will provide you with good feedback, ask questions if something you wrote is confusing, and be available to ask questions if the issue you are seeing doesn’t always make sense to you. I’ve worked with many different developers along the way, and the good ones were the ones that gave me feedback on my work. As long as they aren’t making you feel stupid (which I hope isn’t happening), listen when they bring you feedback. It will only make you better.

Another reflection that I have is that you think AI might be making your reports better, but maybe not? It’d be interesting to experiment. You could spend a week writing your own tickets and just seeing if they end up being confusing for developers. You might find that what you are writing already makes sense and the AI is not needed. I do understand if you are writing something and it’s just not coming together like you want it to read to other people. In that case, I’d say let AI review it.

One complaint I have with AI is that AI, many times, can strip out the unique voice of the speaker. Yes, it may sound professional, but sometimes, you can be just as clear about what a piece of functionality is doing without AI injecting a professional tone, if that makes sense.

You’re probably doing better than you realize. And, if you are working with good people, they will let you know how you can improve.