Hey folks, API Testing is quite important in building quality products and engineers have been implementing frameworks to build confidence in API development process by employing test libraries for assertions.
I wanted to specifically ask API Test framework builders, how they tackle Schema validation - How often do you do it ?
Do you do it
For every endpoint?
Only on most impactful endpoints?
You don’t do it. Or have never done it and don’t even see the value in it?
You do it just for endpoints that implemented recently, as you started lately?
Relevance and extent of doing schema validations. Did you get any real benefit if being used?
What are your views?
I always say that schema validations are critical. They’re your first line of defense against breaking changes in your API. It might not be perfect or the only layer of protection, but it’s certainly a good one. I know schema validation is a hard requirement in many organizations, including mine, which I agree with.
That’s why I created a plugin for Cypress, cypress-ajv-schema-validator. It leverages AJV as the core engine, providing a number of features to facilitate validation against Plain JSON schemas, Swagger documents, or OpenAPI documents. It presents the violations in a user-friendly way directly in the Cypress log, browser console, and User Interface.
It also offers full integration with Gleb Bahmutov’s @bahmutov/cy-api and Filip Hric’s cypress-plugin-api plugins.
In my experience, schema validation occurs in an XML file for checking the request and response, as well as the model and services file. Additionally, in the Enterprise Architect viewer, you should be able to see the exchange model fields request and response in a proper way to validate the fields.
Also, I have been using SoapUI and ReadyAPI for API testing, so the endpoint is configured and given by the development team. Yes, absolutely. If you are going to check an XML schema, you can easily find the observation message.
If something is not working, we can create a bug and report it to the development team. So, Schemas are always helpful for me to spot the bugs.
A lot will depend on usages of the the API. If internal, you can do what you want. However I always look at ours as ‘consumer driver’ - so we need a contract.
PACT is good for this. With recent work on upgrading everything we do, I have got the team to produce the API schema first so that teams on either end understand that contract.
We then ‘DO’ verify schema on all endpoints throughout the development phase to ensure teams are complying with it.
Tools - Swagger for OpenAPI documentation, then Postman for validation - good thing is you can import to a collection.
All seems smooth at the mo and has caught many issues
In my context using Pact does not make sense yet, though I want to.
Indeed, usage matters a lot. In my context, I use an external API provider as a service, different schemas are quite complicated and changes with time, data and parameters.
