I heard about this type of testing fairly recently and still have a few things in my mind which are not clear about it. Seems like there is a lot of mock involved. So if I understood correctly when you’re testing the provider you would mock the consumer and when you’re testing the consumer you mock the provider.
Still seems kind of abstract a bit. Has anyone worked with contract testing?
If you could explain this to me like I’m 3 years old toddler I’d be most grateful!
There is some Fear, Uncertainty and Doubt around contract-testing. Don’t be afraid on new techniques but do learn how to use them appropriately . You absolutely do not mock out both ends that would be an anti pattern.
Contract Testing is a very useful and well proven technique for adding assurance that two (or more) services continue to be compatible without having to deploy them both at the same time for integration or e2e testing.
The idea has been around since the 1980’s but has become popular with growth of microservices and release of “pact .io” in 2013.
A challenge with microservices is that it can become prohibitively slow and expensive to deploy very large numbers of services to multiple test environments. Also you want to enable teams to operate in parallel, team A might be creating a service dependent on team B before team B have finished.
The way it works is by defining a contract between the services. A contract is simply an API endpoint or json schema for a message. This contract can be generated by the provider (provider driven contract-testing) such as swagger/openapi spec or can be specified by the consumer (consumer driven contract-testing) ala pact.
In either case:-
The consumer, validates it’s code can handle messages following the contract ( this is where mocks are used).
The provider, validates it’s code only generates messages following the contract. (here a real service is used with matching or schema validation).
There may also be a broker - which distributes the contracts, and has information about which versions have been tested by both consumer and provider.
So you see both parts get testing but testing is separated. It isn’t quite the same as replaying traffic but it has some similar concepts.
An acute tester will spot the possibility of gaps in the possible variations of messages still meeting the contract. But you can cover a few variations and that possibility also exists for integration testing.
Contract-Testing is also likely to be combined with some targeted integration tests, smoke-checking in production or other risk reduction techniques such as canaries.
So for sure it’s a big change in the approach to testing, and some people feel a loss of control with the lack of e2e testing but control is an illusion and it’s a proven technique which has enabled a number of organisations to succeed.
The way we implemented contract testing is a bit different, since we wanted it to align it with the API tests that we already have written using mocha + chai.
Before we even test any logic using the above framework, we add contract tests to the suite as step 1.
What the test does is, it acts like a client and calls every endpoint, then matches the response with the openapi/swagger model that we test against.
There are some really good libraries to check if the openapi/swagger model matches the response, so we didn’t have to build this functionality.
We have been using this for the past few years and have been so valuable, especially in enforcing the API specs.
Hello, Mirza!
I described what is contract testing in my talk - Practical Contract Testing. I hope the pictures and use cases will be useful.
There are also wonderful videos from Pact creators on what is contract testing and its use cases.
Personally, I worked with contract testing for 3 years (from idea to implementation for the large-scale organization) - so I can describe the real pros and cons of this approach if you are interested.
Note: I’m biased towards Consumer-Driven Contract Testing (CDCT) and Pact - there are other ways of doing it.
One way I like to think it’s that CDCT is a way of creating unit tests for your integration points (I/O code), e.g. Client objects (on Consumers) and Controllers (on providers).
On the Consumer side, the Given part of the test is the configuration of a mock server which will have a specific artificial behavior. You then configure your Client object to interact with it and drive the development of the Client object based on that behavior.
This will check the Client object and as a by-product generate an interaction contract.
One then can use this interaction contract to drive the development of the provider, since the Requests can serve as When and the saved Responses are asserted against the real Responses, thus, part of the Then - the Given part would be the necessary configuration to isolate the Controller from non-I/O code.
One could create unit tests for Consumers without the generation of the contract without a problem. And one could create unit tests for Providers without any input from the Consumers, but then we wouldn’t have any integration check between them, forcing the Consumers to adapt to whatever API is provided (which is what we do in public APIs).
when you’re testing the provider you would mock the consumer
I wouldn’t say this. The contract is the behavior exercised by the consumer, it’s not per se something controlled by the developer of the Provider side. The Provider developer could use mocking to do that isolation from non-I/O code.
Would love to hear your pros and cons Alexander. I will be running my “Introduction to contract testing with Pact” session again in February. Would be great to share these with the attendees and the community.
Anyone coming here who wants to learn more about this topic by comparing Integration Testing and Contract Testing, applying the concept through interactive activities then come and join my session in February on Introduction to Contract Testing with Pact