APIs or services that cause problems when testing?

I have heard from my clients that when they integrate with FedEx services and test their applications with FedEx Sandbox, it causes testing issues. The test data is not available, services are slow to respond, and intermitently not available. This means that testing typical scenarios sometimes takes days instead of hours.

What other services or APIs case you problems during testing and why?

2 Likes

I once hat something similar with with the PayPal sandbox.
The availability and data was fine.
But we accessed it via Browser automation and they changed the UI from every 1-2 month.

Was not a big hassle, but on regular base our automation coded need to be adapted to match the new DOM locators.
I did not researched, but I wonder if there was way to get informed about new deployments.
It always surprised us.

2 Likes

I dont work with backend, but the testers using the Safesforce sandboxes complain all the time of fragility. A thing that scares me is that we don’t actually have the time to set up and “canary” lamps.

I/We have a canary in our client test environment, but to be fair it’s not that reliable either, and that’s testing an internal API surface. At least it’s performant, but most of the time sandboxes are not performant seems to be the issue, because we keep loads of simulation data in the sandbox, and this perhaps impacts any new record sync disproportionately. I luckily don’t deal with it, but I can see the tech debt pain in my colleagues eyes.

2 Likes

In my current role, I tend to use the third approach mentioned in the linked piece, building mocks or stubs if the vendor sandbox/integration environment is problematic.

We’ll periodically do manual verifications against the sandbox/integration environment, but for most of the development life cycle, we’ll be running against internal mocks and stubs. Most of the time we’re not changing our integration, so once we’ve got something working, the risk around the integration is low.

The only place where this becomes really challenging is the performance case, but for those cases, it’s important to get numbers from the vendors, SLAs, etc, and engineer around that, rather than hoping their non-prod environment is going to be analogous to prod in terms of performance.