- Why isn’t contract testing more widely adopted? It appears to be a valuable tool for identifying issues earlier in the development cycle.
- Typically, who takes charge of bi-directional contract testing, developers or SDETs? It seems like it necessitates access to code to be effectively executed, doesn’t it?
2 Likes
Very good question and points.
Unfortunately I cannot answer them since we do not use contract testing at the moment.
While I can recommend this Contract Tests with Pact since this is where I first had contact with contract testing
- I think it goes a little like this: the number of companies who could benefit are limited to those who deal with sufficient services that get updated often, then it’s further limited to companies where there’s awareness of contract testing (haven’t experienced many), and then further limited by companies usually only being aware of contract testing through Pact and being scared off by it being difficult to integrate and get return on investment - an alternative could be what you’ve mentioned - bi-directional testing (such as with Pactflow).
- Typically set up by developers, from what I’ve seen, but test specialists could do it too if they’re comfortable with CI/CD configuration and scripting. I think it should be a collaboration, though, as everyone will need to understand how it works, what it’s for, how to solve problems with it, and write/maintain the provider and consumer tests.
Hey hey!
First question:
Contract testing is a difficult practice to implement and scale. To make it work, you need dedication from the management and engineers.
Contract testing is all about communication and collaboration. If communication is broken - the contract won’t help with it.
Contract testing requires changes in the toolset and development processes. It can even slow down the delivery at the start - when engineers have not used to this new layer of tests.
Contract testing - is an additional “safety net” out of other types of tests. So there is no reason to start work with contracts if you don’t have unit, component, and e2e tests set up in place.
Contract testing is not a “magic pill.” In a nutshell, contracts check only the format of the messages—nothing more. Carefully implemented contract tests and component (service-based) tests can substitute some (but not all) end-to-end tests at the API or UI level, allowing you to concentrate more on the happy path at the E2E level.
Sp contract tests are practical when dealing with hundreds or thousands of services. And it takes time to get the results - from 3 - 6 months to a year.
Implementing this process at a scale of three to five services is a bad idea.
Second question:
If you want an effective process, you need to put developers in charge of contract tests the same way they are in charge of unit tests. The one who changes an API (either producer or consumer) should change the contract first, then wait for reviews from other dependent services, and only after that change an implementation.
SDETs can help with covering legacy services with contract tests.
Test engineers can even help define and add new contracts. But to write tests, you need a solid background in programming.
- Maybe people know it’s useful but find it hard to implement, time- and resource-consuming, don’t have experience or understand its benefits, or have other things to focus on.
- It doesn’t really matter who does it, as long as they know how to do it well. It’s more about getting the job done than who does the job. Knowing how to look at and understand the code is important, especially when making automated tests. It’s all about having the right skills, not the job title.