How do you embed test environment SLA in vendor commercials?

Discussions
, , , ,
1

I asked the question below 3 years ago and I fear I didn’t get a satisfactory answer. I’d like to reopen the discussion and attempt a possible solution, which I hope the community can critic.

SCENARIO

Your organisation has procured an application which they plan to integrate into their technology stack and put live. Obviously after testing, of course.
Fortunately, a test environment is included in the commercials. After the initial implementation, the test environment remains to support the testing of enhancements and integration of peripheral applications.
Suddenly the is a problem which can only be attributed to the test environment. You report this to the vendor. If you’re lucky the ticket goes to the bottom of a ridiculously long backlog. You are told that Service Level Agreements (SLAs) are in place for the production environment incidents, but there is no procedures for test environments.

Having missed the initial opportunity to influence the commercial, any suggestions as to how to retrospectively introduce SLAs for testing issues or bug fixes?

SUGGESTION

As an option, how about utilising the cost of the bug on the organisation, regardless of where it manifests? For example: If a $20K per day cost of a production bug, to the organisation (perhaps in unrealised revenue), is classed as a P2, then a similar unplanned cost of $20K in testing (perhaps people twiddling thumbs and not working) should also be a P2.

Let’s discuss…

2

@gokef15,

An important point to be made here is that testing-level environments all too frequently find no place in contracts, but to me, they are very much a requirement for delivery. I love how you give actual business cost of downtime in testing within the same severity model as production so that the discussion is shifted to “blocked value” versus mere “non-prod inconvenience.”

So an alternative I can think of is to put the SLA terms on production, but with a wider window for test environments (e.g., P1 prod = 2 hrs; P1 test = 6 hrs). Back it with a clear escalation path, and better yet, some kind of addendum or a service schedule that explicitly refers to non-prod. Even if you cannot manage to get the whole contract rewritten, this writing surely holds the vendors accountable.

2 Likes
3

I work with big contracts €100 mill group, in those we have stated SLA’s for all environments - including dev and test. I’m actually on the provider side, but same suggestion. count it in from the start. not that it helps you much now, sorry :smiley: .

Some of framing I would consider are:

  1. time to resolve production incidents, that may require the test environment.

  2. effectiveness of the coming releases, ie the slower test phase the longer time to market, less features pr release