UI Test Automation where? - Fully mocked Dev vs. Integration Environment

🗄️ Archive
1

Hi,
is there a best practice or framework to decide in which type of environment to run automated UI tests?

In my company we have three environments:

  • Dev
  • Integration
  • Pre-Production

In Dev our developers have mocked most of the APIs and required test data can be easily created, too.
The Integration environment is stable, the effort to create test data is ok, but requires some effort for maintenance.
In Pre-Production the effort to create test data and maintenance is huge.

Deployment to any environment is very little effort (< 15min).

Considering only the effort, it would make sense to run the tests mostly or even only in the mocked dev environment.
However in my experience a lot can go wrong once you move to integrated environments.
I would trust those test results much, much more.
But I also never worked in a company where we had the opportunity to run automated UI tests in Dev.

So my guess is to do the following:

  1. Write all tests in a way that they can potentially run in all environments.
  2. Run all tests in Dev
  3. Run all tests in Integration
  4. Run only critical tests and test for error-prone features in pre-production

But what is best practice?

Is there a good read on this topic?

Thanks,
Dennis

2

I would say that the UI tests should be focusing on the UI itself so you could have all your UI tests in Dev…
then… you need to make sure the that the bits that are mocked are properly tested in isolation and also how to test the integration between the services.
It can be painful if you have to write your tests in different way in order for it to run in different environments, so yes ideally your tests should be written in a way that it would make it easy to change environments.
If you have no way of testing the integration without using the UI, then probably building a smaller set that focuses only on those aspects would make more sense.
Regarding topic number 4… I would say ideally everything has should have already being tested at that point… so you should only focus on re-running some sort of checklist(smoke test set) that gives confidence that everything is working well together…

hope it helps…