Ask Me Anything: API Testing

Both of these types of testing have their place. Contract testing is done when you’re consuming an API from a 3rd party generally (whether it’s another team, or company). These are good to help you both (provider and consumer) ensure that changes being made by the provider of the API don’t break things the consumer(s) need/use. This is down to things like datatypes as well as the structure of the responses and response code/response body pairings.
Functional tests actually make sure things still function the way they used to! A provider of an API may or may not make any changes, but we need functional tests to make sure we didn’t screw things up either.

1 Like

@lgibbs captured sketchnotes from the session too!

1 Like

I see those as one and the same, so I’m not sure how you’re differentiating here.
I answered a question during the live AMA about separating tests between API and database, for instance, if that’s what you’re getting at here?
For that, essentially “it depends” of course, but you need to see what your database is doing vs what the API is doing, and verify they’re doing their things correctly.
For instance, if a DELETE from the API isn’t actually supposed to delete from the database (it sets a flag in the record, a type of “soft delete”), then you’d want to check both the API and database for that operation. But if it’s really supposed to remove from the database, then an API GET for that same item you deleted would suffice in my opinion.

I think they both have their merits. I think there are some details here I’m not getting that would better inform an answer. I hate to say “it depends” but it really does! Feel free to provide more detail and I’ll try to answer better :smiley:

This depends on the API you’re testing. It looks like postman does support SOAP (which I didn’t think it did), so that might be a good place to start. There are a lot of great tutorials out there, and it’s a pretty well used product so you can get support (on The Club, slack, or twitter) if you’re stuck.

I have no idea lol I wish I could time travel… not sure what you’re referring to here unfortunately

That’s something you’d need to Google, I really don’t know. I would assume not - you might need to use SQL Server Management Studio or similar to do that

First, I think that anyone that works in software is very technical, we just have different specializations!
I think if they’re not familiar with how APIs work, I’d start there. There are some great courses on The Dojo for that like https://www.ministryoftesting.com/dojo/courses/introduction-to-http and https://www.ministryoftesting.com/dojo/courses/the-building-blocks-of-the-internet-mark-winteringham

Once they have that familiarity, I think folks can do some tutorials with a tool like Postman to become familiar with that and how it interacts with the APIs.
There’s a lot of value in using the same tools as the dev team, so having one of the devs run folks through how things work with your APIs in Postman would help, too.

I’m not super clear on the question here, but if you’re just looking to make sure tables are populated and there’s no existing API endpoint, you can query the database (assuming you have access to do so)…

Mountebank is used for service virtualization, so not really testing all services completely - assuming you’re using something like supertest with mocha to connect to the APIs…
Pros are you’re able to test the various APIs in isolation
Cons are if you rely on Mountebank too much, you might miss critical issues. It’s great to test in isolation but you need to see how they’re working together to get the best picture of your APIs health from these tests.

For contract testing, PACT is an industry standard I think, and is supported in multiple languages.

I have never tested gRPC but I’m sure the community has some great ideas!

I’m not familiar with proto messages, so I’m not sure how mocking them would differ from mocking otherwise. Maybe the community can help?

I would approach this the same way I would testing microservices which act the same way in some cases.
Test the APIs in isolation, mocking what needs to be mocked.
Then testing user or process flows without mocking, to ensure the connections work still etc.
As for the tool to run the tests, that depends on what tool you’re using to write them. They can be written in any language, really, and lots of tools exist to make that easier for folks that don’t do much coding. For the non-coded tests, that depends on the tool (setting up “playlists” of tests etc).

If you have consumers of your api, this is very important to check (and hopefully have consumer-driven contract tests that check that).
Otherwise, the schema may not matter as much as the data. For instance - does it matter that what used to be a String value is now returned as a DateTime value? Maybe not, as long as the data is the same.

1 Like

I’m not sure the question here - basic API testing would be testing each available method (GET/POST/PUT/DELETE/PATCH/etc) for a given endpoint, yes.

It depends on if anything specific is being sent in headers. If I’m not expecting anything to be sent in a header, I generally ignore them. But if it’s something the API should be setting, it should be tested.

I have never used any of those tools :confused: the tools I use and like are:

  • Postman (I use this for manual testing - ad-hoc and exploratory testing)
  • Swagger/OpenAPI (ad-hoc testing)
  • RestSharp with C# and NUnit or XUnit (automation)
  • Supertest with JavaScript and Mocha (automation)

No. Unit tests and API tests cover different things. I think unit tests would still be needed to catch things before you get to the API level. You want to make sure the unit itself works well before integrating with others (inside its own codebase or another api)

You can’t rely 100% on any type of tests at any one level (even if you could somehow get 100% test coverage) - it’s just a fact of life there will be bugs if it’s software!
API tests will not tell you if the javascript on the frontend is working, for example. Even if you had somehow 100% API test coverage, you could still have issues with the frontend because it’s different code, and it interacts with things in various ways.

Absolutely and you should! You can do basically all of the same types of testing against an API as you would any other kind of app - even usability testing! If there are consumers of your API, you can do usability testing to make sure it’s easy to use and understandable.

Yes, as we have consumers of our API that’s kind of part of our usability testing (that I mentioned above). If something goes wrong, we want them to know how to fix it. So if it’s a 400 (which is Bad Request) we want them to know how to fix the request so it’ll work right next time.
Definitely try to avoid as many 500s as possible (those can crash things!). So testing for those “unhappy path” scenarios is a great way to find the accidental 500s, which are just error scenarios not specifically handled by the code.

There maybe more (need to google it!) but the ones that are most tested, by me at least, are:

  • GET
  • POST
  • PUT
  • DELETE
  • PATCH