Mountebank is used for service virtualization, so not really testing all services completely - assuming you’re using something like supertest with mocha to connect to the APIs…
Pros are you’re able to test the various APIs in isolation
Cons are if you rely on Mountebank too much, you might miss critical issues. It’s great to test in isolation but you need to see how they’re working together to get the best picture of your APIs health from these tests.
For contract testing, PACT is an industry standard I think, and is supported in multiple languages.
I would approach this the same way I would testing microservices which act the same way in some cases.
Test the APIs in isolation, mocking what needs to be mocked.
Then testing user or process flows without mocking, to ensure the connections work still etc.
As for the tool to run the tests, that depends on what tool you’re using to write them. They can be written in any language, really, and lots of tools exist to make that easier for folks that don’t do much coding. For the non-coded tests, that depends on the tool (setting up “playlists” of tests etc).
If you have consumers of your api, this is very important to check (and hopefully have consumer-driven contract tests that check that).
Otherwise, the schema may not matter as much as the data. For instance - does it matter that what used to be a String value is now returned as a DateTime value? Maybe not, as long as the data is the same.
It depends on if anything specific is being sent in headers. If I’m not expecting anything to be sent in a header, I generally ignore them. But if it’s something the API should be setting, it should be tested.
No. Unit tests and API tests cover different things. I think unit tests would still be needed to catch things before you get to the API level. You want to make sure the unit itself works well before integrating with others (inside its own codebase or another api)
You can’t rely 100% on any type of tests at any one level (even if you could somehow get 100% test coverage) - it’s just a fact of life there will be bugs if it’s software!
Absolutely and you should! You can do basically all of the same types of testing against an API as you would any other kind of app - even usability testing! If there are consumers of your API, you can do usability testing to make sure it’s easy to use and understandable.
Yes, as we have consumers of our API that’s kind of part of our usability testing (that I mentioned above). If something goes wrong, we want them to know how to fix it. So if it’s a 400 (which is Bad Request) we want them to know how to fix the request so it’ll work right next time.
Definitely try to avoid as many 500s as possible (those can crash things!). So testing for those “unhappy path” scenarios is a great way to find the accidental 500s, which are just error scenarios not specifically handled by the code.
There maybe more (need to google it!) but the ones that are most tested, by me at least, are:
Like any other large application, do a risk assessment against each endpoint:
How often it’s used
how often it’s broken
how critical it is to business
how critical it is to consumers of the API (websites and other APIs alike)
Use level of risk to determine priority
This varies depending on what the API has as far as data. Currently, I create new data objects to test with every time for the tests that require data in a request/response etc. It’s not really “mock” data, it’s just test data
The Dojo has some great courses (answered above) and I’m coming out with a Dojo course soon on automating API checks with RestSharp!
I’m so sorry I missed the notification for this.
If it’s of any use:
When I was working on GraphQL, our frontend was built directly off those GraphQL queries.
So, we structured our tests to map exactly to the queries / mutations that were being called by the frontend.
That’s what made the most sense in our context - if you start going down the road of every different possible query / mutation, I think the possibilities start to become endless!