Many of us who are working in the software testing domain deal with writing bug reports on a daily basis. The tools used to document could vary depending on the company and team’s choice but the goal remains the same.“The point of writing problem report (bug report) is to get bugs fixed” – By Cem Kaner. Fixing a bug Depends on how effectively and meticulously the bug report is documented. The Developer working on the bug makes a decision of whether the bug is reproducible or not based on the information and details provided in the bug report.
Bug Reporting Is A Skill
Anyone can file a bug to report an issue. But it takes great skill to report a good bug. I see a Quality Engineer writing a bug as a great information presenter. By writing a good bug he or she carries a goal of helping the developer and product team to understand the problem and thus make an effective decision of fixing it. So what is a good bug and how to distinguish a good bug from a bad bug? The below characteristics uniquely identifies a good bug.
Clear Title: A good report always has a clear and concise description that describes the problem. The title description should give the reader some idea as to what the problem is about or which area of the functionality could the problem be related to. Sometimes, if we know for sure what feature or functionality the bug could relate to, Prefixing it with feature name or functionality name could be a good idea to help during bug triaging.
Helps Visualize the problem: A good bug report helps the developer or anyone reading the bug report visualize the problem. The steps described should be clear and easy enough to understand and reproduce the issue. The Tone set in writing the report should not be commanding.
Carries the Objective: The good bug report carries the objective and purpose throughout.
The language and tone used should be simple. The steps should be documented in simple meaningful sentences. Being specific with clear details is highly important. (Eg: Test login used What information is used to fill in the form etc… ). This information helps the developer in reproducing the issue quickly.
How & When: Reproducibility of the issue plays a key role in deciding whether the bug should be fixed or deferred. A good bug should always communicate how the test is conducted and at what point did the bug occur. Clear bullet-pointed steps to help narrate the how part. A bug report should also have the answer to When and where the test is conducted. Details such as build number Test environment browser and operating system used while the test is conducted are very helpful in reproducing the issue.
Expected Vs Actual Behaviour: A good bug report communicates clearly the actual identified behavior while the test is conducted along with the expected outcome of the test
A Sample Bug Report
[ New Customer] Admin user is unable to add a new customer
Browsers Tested: Chrome (Version 78.0.3904.70)
Internet Explorer 11
Operating System(s): Mac OS High Sierra, Windows 7
Build Number: 15.7.1
Test Url: XYZ
Test User used: testuser1/ Passw0rd
Steps To Reproduce:
-
Log in as an admin user using the Login details provided above
-
From the top navigation menu click on the customer tab and wait for the dropdown menu options.
-
Click on the add new customer option from the menu
-
From browser dev
-
Fill in all the mandatory fields with the required information below and hit the SUBMIT button.
Custmer_FirstName: User_123
Customer_LastName: Smith
CustomerAddress: 50 Fremont Street Californina - 95037
Customer contact email: automationuser@test.com
Actual Result:
A warning message is displayed on the UI Only on the chrome browser.
The issue does not appear on Internet Explorer.
The warning message Reads as " Exception adding user User_123".
The user is not added to the system based on the backend query ran.
Select * from Customer where First_Name = User_123
Expected Result:
Admin User should be able to add a new customer and success message should be displayed at the end.
Additional Information:
Screenshot Attachment
Please find the recording of the flow attached. (RecordIt video)
Log trace: Log trace from the AddCustomer Service…
What are Intelligent bugs
As humans, we often relate intelligence to our own cognitive ability. When we think of intelligent animals and insects such as elephants or honeybees we associate them with demonstrating human-like aptitude such as the ability to solve problems effectively. In a similar manner Bug reports that provide better and helpful information leading to effectively resolving a problem can be thought of as an intelligent bug.
Writing Intelligent Bugs
In order to write effective and intelligent bugs, the test engineer has to do some research and should make a note of some key pointers to report by answering the questions below.
Is there any similar bug that is already reported?
By asking this question and spending little time scanning the existing issues duplicate
Bug reports can be avoided. Duplicates often confuse and annoy the developers.
Can This issue be reproduced on other browsers or environment(s)?
Verifying if the issue is reproducible on other browsers and on different environments helps to isolate browser and environment issues.
Does the issue occur every time the steps are performed?
After an issue is identified do not rush to conclusions and jump to reporting. Try to repeat the steps multiple times and see the behavior is consistent. This helps in avoiding intermittent issues.
Is the issue persistent upon a page reload or refresh?
Sometimes a bug or issue observed resolves itself upon a page refresh or when the browser cache is deleted. In such cases ensure that the issue can be reproduced on a browser’s incognito mode.
What build version is the issue occurring?
Make sure to always keep a note of the browser version and build version the test is conducted on.
Do I have a screenshot of the issue or errors observed?
Capturing the key screenshots with error messages will help us keep a record of the evidence to attach to the bug report.
Can I record the steps followed to get to the error?
Video recording can speak a thousand words. Especially when there are lots of steps to be executed to see the issue or error occurred. There are screen recorder tools such as RecordIt that can help record the test execution.
Can I capture a stack trace or error log?
For application crashes, Capturing the system logs and crash log dump is really helpful.
Keeping the developer console open while performing the test helps understand the service request calls being made and if there is an error code being thrown back. Expanding the network called the payload sent by the UI and the response retrieved are clearly displayed. This information can help narrow down to problematic service.
Are the backend services up and running?
Ping tests are the best way to ensure that the backend services are all up and running. Often in companies, There is automated tests setup that runs and ensures the backend service ping tests are working well. Another way to ensure is Developer Console. By opening the developer console tool, we can monitor the backend service call made and the response.
What log message and JavaScript console errors are tracked in the console when the error occurs.?
It is very important to keep a close eye and track the Javascript errors in the Developer console of the browser. These Javascript errors in the console may be key evidence and relate to the issue being observed.
Handy Tools And Other Helpful Ways
Mentioned here are some ways and tools that can empower a Test Engineer to engrain Intelligence into the Bug reports.
Developer Console: While testing web Applications, when an issue or odd behavior on the User Interface is noticed, Opening up the Developer Console helps in digging the issue deeper. Developer console can be used to inspect elements, capturing console errors. To Open the Developer Console While On the Chrome browser, right-click and choose Inspect. Click on the Console tab to monitor the console errors.
Preserve log: Whenever you load the page, everything in the console is lost. If you still need the logs on the page to reload or navigation, you can check to Preserve log checkbox in the Console tab.
Disabling cache: Can be done either by going to the File menu and selecting New Incognito Window which will not only use a fresh cache but also won’t use any cookies you may have stored or you can go to the Network tab and check the Disable cache checkbox.
Logs: logs are one of the main objects for analysis. Log trace plays a crucial role in helping the developers identifying the root cause of the issue. Knowing where to go to search for the logs and which log to observe for a specific service is a great knowledge to have. One can obtain the knowledge on logging and which logs to monitor by communicating with dev partners or architects. When there is an Error at UI level you can capture the step/error details from log After completing your testing, cursory check on logs for the exception would throw any memory leak / out of bounds exceptions, timeouts.
Mapping of the Backend Services:
Situations where there is no User interface involved and when the functionality is purely backend. It is crucial to establish a mapping of backend services. Information such as, What is the communication flow between different services and the integration areas as well as the Log files that these services talk to can be detailed into a reference map. This helps the Test Engineer visualize the Service call interactions and making searching for logs easier.
Backend DB Validation:
The database is a key component of any application. Everything is in harmony when the User Interface talks to the Database in the desired manner. If there are problems, we will experience inconsistent results, security threats, and performance bottlenecks. Sometimes the Operations that we perform from the UI such as adding a new customer or deleting an item, may not be updating the database records resulting in inconsistent test results. In such cases, including a backend Database query validation help us identify the inconsistencies. Knowing the Database used and the tables that the application is interacting with as well as the queries to write help in many ways.
Significance Of HTTP Status Codes:
It’s also important to understand that status codes are defined to be potentially applicable to every HTTP resource. HTTP response status codes indicate whether a specific HTTP request has been successfully completed. Responses are grouped into five major classes. The first digit indicates what kind of response it is, and the other two indicate specific handling for the response. Different kinds of responses defined are as below.
- 1xx - [Informational]
- 2xx - [Successful]
- 3xx - [Redirection]
- 4xx - [Client error]
- 5xx - [Server error]
Conclusion
The intelligence of a bug lies in keeping the content short and sweet, yet informationally rich. It makes our engineers happy and efficient. Test Engineers are the eyes and ears when it comes to weeding out issues, and just like in any good relationship, communication is key.intelligent bugs help foster this communication between the test engineer and the developer. This way we can hunt and resolve bugs in the most efficient way — together