Am I understanding the Contract Testing the right way?

I heard about this type of testing fairly recently and still have a few things in my mind which are not clear about it. Seems like there is a lot of mock involved. So if I understood correctly when you’re testing the provider you would mock the consumer and when you’re testing the consumer you mock the provider.

Still seems kind of abstract a bit. Has anyone worked with contract testing?

If you could explain this to me like I’m 3 years old toddler I’d be most grateful!

5 Likes

I would say you got it pretty much @mirza but to be honest I did not really use it until now.

You can have a look here, this is where I first heard of contract testing

Hope it helps

5 Likes

I think I started that course once but I continued past the introductory lesson, might be a good time to resume it, cheers! :nerd_face:

Hmm, you are also on the University? That is nice.

1 Like

There is a meetup about contract testing tomorrow eve (EU-time): Consumer Driven Contract Testing with Raquel Bautista-Garrido | Meetup

It might be the place to ask all thy questions :slight_smile:

1 Like

Thanks a lot Kristof, I’ll definitely sign up for that meetup!

1 Like

There is some Fear, Uncertainty and Doubt around contract-testing. Don’t be afraid on new techniques but do learn how to use them appropriately . You absolutely do not mock out both ends that would be an anti pattern.

Contract Testing is a very useful and well proven technique for adding assurance that two (or more) services continue to be compatible without having to deploy them both at the same time for integration or e2e testing.

The idea has been around since the 1980’s but has become popular with growth of microservices and release of “pact .io” in 2013.

A challenge with microservices is that it can become prohibitively slow and expensive to deploy very large numbers of services to multiple test environments. Also you want to enable teams to operate in parallel, team A might be creating a service dependent on team B before team B have finished.

The way it works is by defining a contract between the services. A contract is simply an API endpoint or json schema for a message. This contract can be generated by the provider (provider driven contract-testing) such as swagger/openapi spec or can be specified by the consumer (consumer driven contract-testing) ala pact.

In either case:-

  • The consumer, validates it’s code can handle messages following the contract ( this is where mocks are used).
  • The provider, validates it’s code only generates messages following the contract. (here a real service is used with matching or schema validation).
  • There may also be a broker - which distributes the contracts, and has information about which versions have been tested by both consumer and provider.

So you see both parts get testing but testing is separated. It isn’t quite the same as replaying traffic but it has some similar concepts.

An acute tester will spot the possibility of gaps in the possible variations of messages still meeting the contract. But you can cover a few variations and that possibility also exists for integration testing.

Contract-Testing is also likely to be combined with some targeted integration tests, smoke-checking in production or other risk reduction techniques such as canaries.

So for sure it’s a big change in the approach to testing, and some people feel a loss of control with the lack of e2e testing but control is an illusion and it’s a proven technique which has enabled a number of organisations to succeed.

7 Likes

Thanks Crunchy, things are much clearer now!

The way we implemented contract testing is a bit different, since we wanted it to align it with the API tests that we already have written using mocha + chai.

Before we even test any logic using the above framework, we add contract tests to the suite as step 1.

What the test does is, it acts like a client and calls every endpoint, then matches the response with the openapi/swagger model that we test against.

There are some really good libraries to check if the openapi/swagger model matches the response, so we didn’t have to build this functionality.

We have been using this for the past few years and have been so valuable, especially in enforcing the API specs.

Hope that helps a bit @mirza

2 Likes

Hello, Mirza!
I described what is contract testing in my talk - Practical Contract Testing. I hope the pictures and use cases will be useful.
There are also wonderful videos from Pact creators on what is contract testing and its use cases.

Personally, I worked with contract testing for 3 years (from idea to implementation for the large-scale organization) - so I can describe the real pros and cons of this approach if you are interested.

6 Likes

Note: I’m biased towards Consumer-Driven Contract Testing (CDCT) and Pact - there are other ways of doing it.

One way I like to think it’s that CDCT is a way of creating unit tests for your integration points (I/O code), e.g. Client objects (on Consumers) and Controllers (on providers).

On the Consumer side, the Given part of the test is the configuration of a mock server which will have a specific artificial behavior. You then configure your Client object to interact with it and drive the development of the Client object based on that behavior.

This will check the Client object and as a by-product generate an interaction contract.

One then can use this interaction contract to drive the development of the provider, since the Requests can serve as When and the saved Responses are asserted against the real Responses, thus, part of the Then - the Given part would be the necessary configuration to isolate the Controller from non-I/O code.

One could create unit tests for Consumers without the generation of the contract without a problem. And one could create unit tests for Providers without any input from the Consumers, but then we wouldn’t have any integration check between them, forcing the Consumers to adapt to whatever API is provided (which is what we do in public APIs).

when you’re testing the provider you would mock the consumer

I wouldn’t say this. The contract is the behavior exercised by the consumer, it’s not per se something controlled by the developer of the Provider side. The Provider developer could use mocking to do that isolation from non-I/O code.

2 Likes

So many helpful answers, thanks everyone! :nerd_face:

you have also this workshop next week in MoT: How to Solve Integration Issues with Contract Testing | MoT

2 Likes

I’ll have to try persuading my managers to consider paying for pro membership! :money_mouth_face:

Next day you have Test Bash Home, June is the month to be pro :smiley:

1 Like

Would love to hear your pros and cons Alexander. I will be running my “Introduction to contract testing with Pact” session again in February. Would be great to share these with the attendees and the community.

2 Likes

Did you manage to persuade your managers? Feel free to reach out if you have any more questions once you start implementing contract testing

1 Like

Anyone coming here who wants to learn more about this topic by comparing Integration Testing and Contract Testing, applying the concept through interactive activities then come and join my session in February on Introduction to Contract Testing with Pact

1 Like

I would love to share the pros, cons, and thoughts on contract testing. Should I do it during the session or in any other form?)

1 Like

If you could post it here. Then I can refer people here during/after the session. Thanks a lot!

1 Like