Assertions for API Responses

[mirza (Mirza Sisic)]

Trying to assert that the JSON body (or XML) in an API response is correct, which approach do you use?

I’ve noticed that some people use full expected response body and compare it to what they get back:

assertEquals(expectedResponse, actualResponse)

This seems a bit too much for me, since the response can change and you’ll have to update your test.

Another approach I noticed is to assert if just a part of the response contains an expected value, like:

assertTrue.contatins("expected-value") 

The second approach seems like it will require less maintenance in the future, especially if you assert on something that you know will not change in the response.

What are you thoughts on this? Any other ideas? I’m really curious!

[kristof (Kristof Van Kriekingen)]

Full body asserts are easy and dirty imho.

I assert on the json values themselves, which results into easier debugging afterwards and cleaner reporting. Let’s see an example (API Testing in Postman)

Entire JSON assert

var completeJsonBody = pm.environment.get("completeJsonBody")
var jsonData = pm.response.json();

pm.test("Check if jsonBody matches", function () {
       pm.expect(jsonData).to.eql(completeJsonBody);
});

Single JSON asserts

var Id= pm.environment.get("Id")
var jsonData = pm.response.json();

pm.test("Check if ID matches", function () {
       pm.expect(jsonData.id).to.eql(Id);
});
pm.test("Check if name matches", function () {
       pm.expect(jsonData.name).to.eql("Kristof");
});
pm.test("Check if age matches", function () {
       pm.expect(jsonData.age).to.eql(29);
});

I just prefer the second option because you can instantly see which test failed and which value is giving you an error. It’s also more clear for reporting to see what tests you do. If on all API request you note ‘body is ok’ compared to ‘Check to see if age matches’ … it says a lot more I think.

@mirza

The second approach seems like it will require less maintenance in the future, especially if you assert on something that you know will not change in the response.

Maintenance wise, I believe it’s cleaner also, because you don’t always have to assert on each value on each request and if something changes or a value changes, it’s an easy fix. + you can actually see what you assert on.

Question: How are you going to compare a GET /lists/ ? What if I make another item that is added towards that list, or someone adds an item manually through the UI?

Looking forward to seeing other people’s answers!

[restertest (Ioan Solderea)]

I would go a little bit different.

First I would check that the format of the response matches the defined schema

Secondly I would check for different values in the Response as @kristof also indicated.

[mirza (Mirza Sisic)]

@restertest That video is getting saved to my Watch Later playlist, I’ll take a look as soon as I’m finished with my meetings!

[ernie]

I use different strategies depending on what I’m testing. I’ll use exact response matching if the response isn’t dynamic, or can be generated strictly from the input (i.e. maybe a JSON object that returns the external ID I provided and a status of submitted or something).

Once it moves to anything dynamic (timestamps, server assigned IDs, etc), I tend to do field level validations, using various levels of checks depending on the risk/criticality. i.e. for date/timestamps, I might check the format and then verify the value is within 5 seconds of the submission, but for some fields I might just check that it’s a string or a number, or ignore them completely.

I’m personally not a fan of JSON schema checks since we don’t develop based on schemas, and the auto generated ones like that shared in the video by @restertest tend to be too loose to be of much value, and I’m likely going to be having to write field level validations anyway. If we did have full schemas as a practice, I’d likely push for contract testing using PACT or similar.

[hiba (Hiba Ghannam)]

I don’t like validating the whole json response even it’s easier to add single assertion for it, I was also wondering about others thoughts so I am thankful for finding this post with these awesome answers! @mirza curious to know what do you recommend after more than a year of posting this.