Advice on assertions to make when creating an API testing framework

(Andrew) #1

I’m currently looking at building an API test automation framework for a series of Rest Services, utilising c#. Coming from a UI-automation background I was after some advise on assertions when testing against an API.

Obviously there are a whole array of different assertions that can be carried out but based on peoples experiences are there certain assertions that must be done? e.g. that the Expected Status Code is returned? Or is this something that can only be ascertained once the API definitions are in place?

30 Days of API Testing Day 8: Explore the API thread on The Club and contribute to the conversation
(Vishal Dutt) #2

We have worked on REST API automation using RestAssured tool. It is Java based library for API testing. It works like in headless mode. User sends the HTTP request using URL and receives the response. RestAssured then allows us to validate response along with its status code and values.

There are different types of HTTP methods types are available in RestAssured like: GET, POST, PUT, DELETE.

Here, we will talk about GET method. If we hit an API for the details of city then response would in JSON format with values like cityName, Temparature etc.

{
“City”:“London”,
“Temprature”:“10 Degreee Celcius”
}

Sample code to retrieve data is as below:

RestAssured.baseURI = “”;
RequestSpecification httpRequest = RestAssured.given();
Response response = httpRequest.request(Method.GET, “/London”);
String responseBody = response.getBody().asString();
System.out.println("Response Body is => " + responseBody);

Validate the Response Status using below code:

int statusCode = response.getStatusCode();
Assert.assertEquals(statusCode, 200);

Now, here you will be interested to validate the values of a parameter from JSON response. Please find below mentioned code to do the same:

JsonPath jsonPathEvaluator = response.jsonPath();
String city = jsonPathEvaluator.get(“City”);
Assert.assertEquals(city, “Hyderabad”);

Hope this information is helpful for you.

3 Likes
(Joe) #3

Hello @tilston1001!

We split up the automated evaluation of APIs. Some automation evaluates the code that implements the API (these might be labeled unit tests), and some automation evaluates the behavior or business value of the API. Both use mocks extensively.

The unit tests exercise and evaluate the classes and methods that make it possible for the API to operate. These might be database interactions, response creation, and others.
The behavioral tests exercise and evaluate the API from strictly an input/output point of view. If the API requested a list of bank accounts, we make the request and evaluate the reply. Through mocks, we can craft the reply to be accurate or to have missing data or to reply with an error. These are the business behaviors. The status code is evaluated here since it is a business behavior.

I encourage you to work with your development team to build the API slowly. Our first deployment to a non-production environment has the API return no body with a status code of 200. Then, we begin adding functionality and behavior. In this manner, our automation was able to keep pace with the development team, and they could use it for regression.

Joe

(Ali) #4

Personally I would often assert on the following, in separate tests

  • Response code - we often check for 200/201, 400 Bad Request, 404 Not Found, 302 Redirect etc.
  • Response body - Json Schema e.g. are all fields of the correct type?
  • Response body - data validation e.g. I’m requesting record 1, does it contain the correct data in all of the fields?
  • Response headers - are we exposing too much information? E.g. server name

Not too sure what you mean about API definitions being in place. For me, this is something that would be discussed at a planning/refinement session with the whole team.

You can discuss some test scenarios with your team and state what your expectations would be (i.e. what you would be asserting on). This helps to iron out requirements and ensures the team is all on the same page and there aren’t any hidden assumptions.

2 Likes
(Conor) #5

Similarly to Ali

  • Status code - Ensure the relevant satus code is returned. Don’t just assert the value, Question is the status code code for this scenario.
    When creating data, a 201 is more appropriate than 200.
  • Response Body - assert the error messages, verify tyst the format consistent, ask if the error message useful to the user?
  • response Body - assert that the data returned is correct based on your response.
    If you want all instances, did you get all instances?
    If you only wanted one instance, did you get the correct one?
    Based on your context, you could verify the data returned from the API against data returned from
    the database
  • Response Headers - perhaps verify the content type. Simple check but can be useful.
  • Response Headers - security - if there is agreement in relation to the security parameters these can often be simple to check and add real value.
    Max age value, X-XSS-Protection Header Values etc
1 Like
(Amit) #6

In addition to the assertions mentioned above, you should also assert for the following:

  • un-authenticated access status_codes are correct, with cookies, etc
  • meaningful exceptions (debug message) in case of error scenarios.
1 Like
(Tu Anh Nguyen - KMS) #7

I have the same answer with alihill Ali. In terms of API testing, I verify the response code, the content of response body and the response headers.
I am working with JavaScript and using Chai Assertion Library to assert tests.
I wonder why don’t you just develop request methods (GET, POST, PUT, DELETE) then use assertion library to verify Response?
I found this https://fluentassertions.com/documentation/ for C#.