Mountebank is used for service virtualization, so not really testing all services completely - assuming youāre using something like supertest with mocha to connect to the APIsā¦
Pros are youāre able to test the various APIs in isolation
Cons are if you rely on Mountebank too much, you might miss critical issues. Itās great to test in isolation but you need to see how theyāre working together to get the best picture of your APIs health from these tests.
For contract testing, PACT is an industry standard I think, and is supported in multiple languages.
I would approach this the same way I would testing microservices which act the same way in some cases.
Test the APIs in isolation, mocking what needs to be mocked.
Then testing user or process flows without mocking, to ensure the connections work still etc.
As for the tool to run the tests, that depends on what tool youāre using to write them. They can be written in any language, really, and lots of tools exist to make that easier for folks that donāt do much coding. For the non-coded tests, that depends on the tool (setting up āplaylistsā of tests etc).
If you have consumers of your api, this is very important to check (and hopefully have consumer-driven contract tests that check that).
Otherwise, the schema may not matter as much as the data. For instance - does it matter that what used to be a String value is now returned as a DateTime value? Maybe not, as long as the data is the same.
It depends on if anything specific is being sent in headers. If Iām not expecting anything to be sent in a header, I generally ignore them. But if itās something the API should be setting, it should be tested.
No. Unit tests and API tests cover different things. I think unit tests would still be needed to catch things before you get to the API level. You want to make sure the unit itself works well before integrating with others (inside its own codebase or another api)
You canāt rely 100% on any type of tests at any one level (even if you could somehow get 100% test coverage) - itās just a fact of life there will be bugs if itās software!
API tests will not tell you if the javascript on the frontend is working, for example. Even if you had somehow 100% API test coverage, you could still have issues with the frontend because itās different code, and it interacts with things in various ways.
Absolutely and you should! You can do basically all of the same types of testing against an API as you would any other kind of app - even usability testing! If there are consumers of your API, you can do usability testing to make sure itās easy to use and understandable.
Yes, as we have consumers of our API thatās kind of part of our usability testing (that I mentioned above). If something goes wrong, we want them to know how to fix it. So if itās a 400 (which is Bad Request) we want them to know how to fix the request so itāll work right next time.
Definitely try to avoid as many 500s as possible (those can crash things!). So testing for those āunhappy pathā scenarios is a great way to find the accidental 500s, which are just error scenarios not specifically handled by the code.
There maybe more (need to google it!) but the ones that are most tested, by me at least, are:
I donāt really think of my approach as a pyramid, just basically the same as a UI testing approach. I find my testing priorities based on risk, use my oracles and heuristics to do my testing, etc.
Containerization doesnāt change much in my experience. The only difference I see is that I can run the APIs locally more easily with containerization
Like any other large application, do a risk assessment against each endpoint:
How often itās used
how often itās broken
how critical it is to business
how critical it is to consumers of the API (websites and other APIs alike)
Use level of risk to determine priority
This varies depending on what the API has as far as data. Currently, I create new data objects to test with every time for the tests that require data in a request/response etc. Itās not really āmockā data, itās just test data
The Dojo has some great courses (answered above) and Iām coming out with a Dojo course soon on automating API checks with RestSharp!
Iām so sorry I missed the notification for this.
If itās of any use:
When I was working on GraphQL, our frontend was built directly off those GraphQL queries.
So, we structured our tests to map exactly to the queries / mutations that were being called by the frontend.
Thatās what made the most sense in our context - if you start going down the road of every different possible query / mutation, I think the possibilities start to become endless!
the tools I use and like are: