30 Days of API Testing - Day 24: Share the best API bug you’ve found


(Duong) #1

I’m a newbie in API testing and have not yet done any reality API testing task. Hope to see many of interesting bugs/defects from the experience members so that I can learn from them :blush:.


(Conor) #2

Can’t think the best but I have found but can share the typical pattern of issues I have found.

  • ‪Test nulls and empty strings, these have a habitat of causing issues ;).
  • Test with Unicode characters, ensure they are correctly saved to the database.
  • Test mandatory and optional parameters, in particular test with just mandatory values.
  • Ensure correct status codes are returned, you will find false positives e.g a 500 being returned as a 200.
  • experiment with data types, if an integer is expected what happens when you use a string.
  • combination errors: test two errors at once e.g invalid account id and invalid customer id, only one error code and error message can returne, is the most relevant status code and error message returned.

Use heuristics

  • CRUD, ensure the API can create, read, update, read, delete and read to ensure the data is actual deleted
  • sequence - change the sequence of your CRUD tests
  • 0,1, many - test the api where there is no data, just one item and lots items

Security
You don’t need to be an expert to do some basic checks
can you use the API without authentication header?
Look at the security headers, are there any settings that could be improved?
If an API requires an account and customer id, test if you have a use a customer if from a different account, can you access the data?

Performance
Performance: have a look at the network tab in chrome dev tools, a typical simple get should return in hundreds of miliseconds, if not explore what it’s doing.
Use a tool like jmeter to some further testing.

Reliability

  • what happens when the service is down before you being to process data or when data is being processed?
  • what happens when intermediate or dependent services are down?
  • what are time-out values?
  • is data queued when service are down?
  • does the service self recover?

(Ilya Lychkou) #3

I’m a newcomer to api testing and I have not found api bug by myself. But I can share my thoughts about possible problems with api:

  • Failed to process valid and invalid data
  • Unused flags
  • Missing or duplicated functionality
  • Invalid response data structure
  • Reliability - difficulty connecting and getting a response from the API
  • Safety problems

(Heather) #4

From the Twitterverse


(Tu Anh Nguyen - KMS) #5

The most benefit of API testing is time effective, and the best API bug I found is the 500 Internal Server Error.
The 500 error code is a generic message that appears when something unexpected happened on the web server and the server cannot offer more specific information.

In my project we use scripts to verify API services so it is really quick to find the 500 Internal Server Error. It saves our testing effort since sometimes the build is dropped because of that bug.


(Trang Ta) #6

I have not worked on real API testing project yet, so I research about API bugs. I summarize API testing find the type bugs:

  • Functionality Bugs: the test looks for missing functionality bugs
  • Reliability Bugs: API testing helps identify bugs pertaining to integration across different systems
  • Performance Bugs: The tests help determine how much traffic the system can handle before being overloaded, and how to expand infrastructure to meet rising demand at the core, but also are very effective at pinpointing weak points in the API
  • Security Bugs

I’m interesting with bug “200 is Not Ok”. Response 200 status code but it is not ok, it is not enough. While validating an API, we should check the entire response in the header and payload.


(Thanh) #7

I am researching API testing, still not apply it in work. Thus, not having any bugs yet so I found some common errors here.
On the other side, through exercises, I met some difficulties in writing an API test, sharing in challenges helped me to solve them, so useful.


(Tuan Tran) #8

I can’t think what is my the best bug so wanna share the common bugs I found:

  • Functionality Bugs: the API return value base on the input and the business logic. But sometime the returned value is not matched the expected. So I used to analyze the business logic and create data for testing.
  • Bug related to the HTTP Code.
  • Bug related to the field name on JSON: It usually come from the typo mistake.

(KMS Nam Nguyen) #9

I have not worked on real API testing project yet, so with my things for the best bug.
Performance: Verify the progress bar in Network tab in chrome console log(F12), the simple method GET/POST

  • How: Any method will displayed the responded times correctly in network tab so we can verify the performance of your API, any responded with long time returns will be show in this tab and we should verify what it is doing?
    Security: I have not worked on real API testing, especially Security testing but with my things, The best API bug come from the exploration test scenarios. The bug come from exploration testing give the best feedbacks to your API.
    Functionality Bugs: The bug related to the data returns are matching/not matching with expectation.

(Thao) #10

I’m just learning API testing. I’ve researched about API bugs as below:


(Joe) #11

We started a new API project and had new definitions for the payloads. As the API grew more complex, the payload definition would change. For a little while, our automated tests would catch payload definition errors. However, when one parameter changed from optional to required, the automated test did not catch it and the API was deployed to our non-production environment. The bug was noticed there.
I think it was a good discovery since we did some enhancements of our automation to review for this kind of bug.

Joe


(Felipe) #12

I’ve been working with API tests for a billing system and some of the most interesting bugs I’ve found are related to negative numbers being accepted in places they should not such as negative invoice totals or allowing a negative credit balance to be saved.