I am interested to know where you consider the start and end of E2E or End-to-End tests?
I’ve seen many approaches to describe test Triangles, Pyramids, Cubes and also the concept Seems or Layers, where tests often cross system boundaries.
I consider myself a “Full Stack Tester”, in fact I think most testers Exploring or Automating systems cross many parts of a software stack with their testing.
From a product quality point of view, I like to think beyond the software stack with the real Start and End being the whole journey. Taking a user from their first interaction with an entity (company, open source community, individual etc) right though to when they use the product for the last time.
This wholistic view is useful form a Quality Process thought experiment, but still leaves me with questions from a Test Automation point of view.
So already in order to codify our testing we need to draw a boundary and define some scope that is smaller then the biggest possible end to end.
This gets smaller still when we start to consider what might be out of scope:
- Interactions with third party systems
- How the user experiences the system
- How the system behaves offline
- How the system works on physical hardware (e.g mobile phones)
- The outputs of the system where they might cost money like shipping a real product
This is just a list of examples and is in no way complete. And yes, some of these have potential solutions, device farms might allow access to physical hardware to automate against for example.
So I come back to my original question and I am interested in your answers. Where do you draw the line in an End to End test, and how much of a users journey do you feel it covers?