Hi,
Has anyone managed to play around with Postman and Restsharp? If so which tool do you think is better out of these/
Thanks,
Rajee
Hey rajee,
It kind of depends on how you want to use it. If you want less-technical people to jump in on your project & do some API tests. I would go for Postman, it’s easier for less-technical people to learn API testing. The learning curve isn’t that huge for Postman compared to RestSharp (from a non-techie point of view).
I think Postman is more popularly used (if not the most popular)! I don’t think it is perfect as such and I haven’t used the other one.
RestSharp is more for automation, whereas Postman can be for manual/exploratory testing as well as automation. So it depends on who will be using it and what purpose you’re looking for. I use both in my testing, depending on what I need to get done.
Thanks for the reply @kristof, @herschel, @g33klady.
I need this for automation testing, where in we have all our expected json created via a tool, which has got the data for POST as well as data for the GET , all I need is to do a get request and check whether actual json matches with the expected .
I need to get a good report on it as its quite a dynamic json , if any mismatch in the values then proper message should pop up…
Currently there is a postman-newman test which does this, but I was trying out restsharp,
the problem is I cant directly do a comparison, because, the expected the fields are very few where as actual fields are everything. so I cant compare the fields directly json to json, also there are issues relating to rouding , when I compare for eg : expected - { “description”:“the value is 28.05” } - actual { “description”:“the value is 28.054” } , when I add logic for handling this rounding issue then looks like he test becomes too slow.
So a bit confused on how to approach this…
Thanks,
Rajee
1 Like
Hi Rajee, I’m not sure how much help this will be to your problem, but for the automated API tests I’m currently working on (Cucumber/Java), we tend to use the jsonunit library in our code for comparing the actual response against an expected response in a reference file. The big advantage of jsonunit for doing this is that if there is likely to be variability in the values returned, then you can put an ignore flag into the reference file. When the comparison between actual and expected is done, jsonunit will only assert that the object is present in the response and not bother about the value returned for it. You can also set your test code to be lenient about the order of the objects returned.
So, you can greatly reduce test fragility in this way. As I generally re-use the same test cases and code between our (stubbed) functional tests and our integration tests we can set the reference files up for the integration test environments so that they primarily assert that a successful API call returns data in the correct structure. This reduces the risk of tests breaking due to data changes on the integration test environments, and so reduces future maintenance needs.
Going back to the original question, I’ve not tried Restsharp so I couln’t comment on whether Jsonunit could be integrated with that.
1 Like
can you share some code?
That gives me a couple of things to think about. Currently, the framework behind my Cucumber Glue code thunks down to bash scripts that execute curl commands and then grep to varying degrees (either just checking the response is correctly formed or, if need be, generating custom regx to test against the output). Would be nice to have it all inside Java.
I need to avoid breaching corporate IPR so can’t post up the exact code but I hope this gives you an idea of how to make it work:
In the glue code file, you need the following import (or use the non-fluent version if you prefer):
import static net.javacrumbs.jsonunit.fluent.JsonFluentAssert.assertThatJson;
Then you need something like the following to find the reference file, convert it to a string, convert the actual response to a string and compare the two:
Then("the response body matches {string}", (String expectedBody) -> {
String expected = <code to find reference file and convert to a string>;
String actual = <code to convert response body to a string>;
assertThatJson(actual).isEqualTo(expected);
});
Then, your reference file needs to be set up something like this, it should assert that the value in the first item matches what’s in the file, but only assert that the second and third item are present, not what their actual values are.
{
“mainObject”: [
{
“firstItem”: “some fixed value”,
“secondItem”: “{json-unit.ignore}",
"thirdItem": "{json-unit.ignore}”
}
]
}
You can do something similar with XML using the XMLUnit library too, that has the ignore value functionality.
Hi sorry I meant if @rajee could share some code.
Also @professorwoozle - RestSharp is a C# library; unfortunately jsonunit is not available in C# but I’m sure there are equivalents
1 Like
Since, in this case, you’re eventually comparing strings, I’m wondering how robust your tests are? What happens if the response contains all the information you would expect, but just put some properties at a different position in the JSON? From your example code it seems like the test would fail, even though the response would contain all the necessary information.
One of the great things about Postman is that it contains a couple of tools already build in its sandbox. For instance TV4. It’s a schema validation tool that helps with asserting if you’ve received the correct response for a given request. The beauty of it is that it doesn’t matter at which position the properties in your JSON response are located. If they are in, the test will pass. It also allows for a lot of flexibility because you can indicate which properties are mandatory and which are optional. Defining those are great and greatly improve the robustness and reliability of your tests. One other nice thing is that you can define the data types of the properties that are returned. For instance, is a field a string or an int or a float or can it be Null. For ints and floats you can even define the range a valid response should fall between.
Postman might look like it’s more friendly for new users at first, but make no mistake about it. It contains a lot of great stuff and things can get rather technical very quickly.
1 Like
In jsonunit, you can set the comparison to run in lenient mode which will stop the assert failing if the ordering of the objects is different to the reference, as long as they are all present.
Hi, I must agree with the previous comments. The only thing I will say that in my experience Postman is ideal for exploratory testing or development of scenarios with dev/BA. It is is quick to get something up and running compared to that of restSharp/restAssured.
However in terms of building automation scripts, both Postman and restSharp (restAssured is what I’ve used) can be used.
At the end of the day, you use the best tool that meets your needs and will be used by QA/BA or even Devs. For example, in my last role we agreed upon the use of SOAPUI because it was already in use and low overheads in terms of technical skill (and could be used by non-technical people). We then modified the test to be run as maven project and then deployed within Jenkins as post deployment tests We had buy in from all key stakeholders.
I kind of agree with @oorupabo on this. Postman is definitely a great tool for exploratory or manual testing. The learning curve on it is rather low.
Technically, you can use Postman for automation using tools like newman. The newman tool will let you run the Postman scripts in an automated CI/CD using command line functionality BUT you still need to use Postman to organize and maintain the tests. For a small suite of APIs this is acceptable but, for pretty much all the projects I have ever worked on, maintaining the test suite gets really difficult over time.
Also, as @oorupabo points out, Postman looks very friendly at first but it can get quite technical, quite fast. I find developers are more comfortable with something like RestSharp. So you can, hopefully, get the developers on the team to help with maintaining/refactoring the RestSharp tests but probably not with complex Postman suites.
I have not used RestSharp but I have used things like RestAssured or just a simple HTTP Client in other languages (Java, Ruby, Python). Then I can use comparison libraries (for unit testing often) and json libraries from the language I’m using. I’m sure there are C# libraries which will help in this area.
Darrell