is there a best practice or framework to decide in which type of environment to run automated UI tests?
In my company we have three environments:
In Dev our developers have mocked most of the APIs and required test data can be easily created, too.
The Integration environment is stable, the effort to create test data is ok, but requires some effort for maintenance.
In Pre-Production the effort to create test data and maintenance is huge.
Deployment to any environment is very little effort (< 15min).
Considering only the effort, it would make sense to run the tests mostly or even only in the mocked dev environment.
However in my experience a lot can go wrong once you move to integrated environments.
I would trust those test results much, much more.
But I also never worked in a company where we had the opportunity to run automated UI tests in Dev.
So my guess is to do the following:
- Write all tests in a way that they can potentially run in all environments.
- Run all tests in Dev
- Run all tests in Integration
- Run only critical tests and test for error-prone features in pre-production
But what is best practice?
Is there a good read on this topic?