In your opinion and from your experience, when the software does not prevent the user from using it a certain way and it results in a non-friendly error message (i.e. SQL error, object not set to reference, array out of bounds, etc.) Do you consider that a bug or a non-issue?
What about when the requirements didnt call this out as part of the design or if you cant locate the original requirements? The result of which Devs say they wont fix and the customer wouldn’t use it that way…
Thoughts?
For me, the non-friendly error message scenarios (being unhandled errors) are bugs. Perhaps unit tests should have picked them up, I dunno, however it is still a defect.
The “customer wouldn’t use it that way” is not a valid excuse in my eye and I always challenge it. Thankfully, I usually have most of the team on my side that it is not a valid reason to not fix something.
Requirements didn’t call that part out… if it is a non-friendly error/unhandled error, regardless of the requirements telling you handle that particular error, it needs handling. If that even makes sense.
TL;DR - don’t let devs use “customers wouldn’t use it that way” as a justification for not fixing an issue, and expect requirements to not pick up random errors. It is actually up to the developer (I don’t want to place “blame” but it kind of is) to know the sorts of errors that can occur from their work and handle them.
Friendly error messages should always be covered by requirements, however possibly some aren’t implicitly defined as this wasn’t a foreseen scenario? In my opinion this would be a bug (based on the information provided) whether it’s just around error handling or fixing the error thrown is another question altogether though.
This appears to be one of those time vs effort vs impact issues. I suspect it’s already in production and the user can recover from it? So if no ones complaining we don’t want to spend dev time on it.
However if you are able to reproduce the issue it’s more than likely a customer will be able to as well. However if someone is pushing back on this as a nonissue you would probably need to prove the impact and try raise the severity of the issue. Otherwise try get it onto the backlog to get fixed on a future release if the product owners are happy with its current state.
This is my experience, not my advise, I have no idea about the product you are working on or the environment you are working in.
If it impacts the user experience, it’s a bug. You may not give it a high priority, but it’s still a bug.
And when a dev says “The customer wouldn’t use it that way”, they are almost certainly wrong.
One of the things I do assume is that whoever wrote a specification document did not write down everything which should happen in the program. I have rarely seen a specification document which said “All error messages have to actually show what the error is about.” That is, however, one of the things I usually test.
What is declared as “This might be a problem” is not always what is written down. That is one of the reasons why we’re testing in the first place, not just to find the desired behavior, but to explore the unknowns. So if we find something which might be a problem, whether it is specified or not, deserves at least a conversation.
Thank you all for your input. It is validating to hear your opinions and viewpoints on this matter and that we agree on this. At the end of the day, requirements are not going to detail every single possible variation. Similarly, requirements don’t typically reference error messages, unless error messages are central to the feature/are of software that is being tested, in my experience. I concur that this is most definitely a ‘bug’ or at least warrants a conversation, or at a bare minimum a ticket for better error handling to be put somewhere in the backlog.
Sometimes it helps us to at least be validated by our colleagues when we find ourselves in situations like this. I appreciate your feedback!