I interview people who apply to work at my company. One of the questions we ask is to explain the test pyramid. I get different answers all the time. So I guess the question you are asking depends on how you define the test pyramid.
Just in the response from so far I see different people saving different things. Some believe the test pyramid is just about automated testing. Others give different levels. Rarely do I see people talk about why I believe the test pyramid is useful.
Is it about automation? I donât think so. I started my testing career in 1998. There wasnât a lot of agile at that time. Almost everything was waterfall. The software developers in my company started adopting agile software development. There was still a throw it over the wall attitude for testing and QA.
Iâd get story 1. Iâd test it. Iâd get story 2. Iâd test story 1 and 2. Iâd get story 3. Iâd test story 1, 2 and 3. After a while I could not test everything. So we started prioritizing stories. The most important stories would get tested first. Out of 500, the QA team might test the top 300 stories but the other 200 stories wouldnât get tested before we got another release from development. The solution I, and many others, came up with was to automate everything. I was automating everything at the end and maintaining the test suite was difficult. But automating a lot of the stories meant we didnât have to manually test everything. Automation helped with regression testing. No pyramid involved.
Always wanting to be efficient, I thought about how can I prioritize my testing? Are their tests I donât need to do? WAIT. There are unit tests. I might have a test that could fail for 7 reasons. I had to gather enough information in the automation to know why it failed. But 4 of those reasons would have been caught at the unit level. So I really only had to code for the remaining 3 reasons the test would fail.
Even better, could I test two classes together and eliminate more reasons my system level test might fail? I could create 2 integration tests. Now there was only one reason my system level test would fail.
This made automating the tests easier. This made debugging failures easier. This made some defects found sooner (fail fast). Then I found out about Mike Cohn and the âtest pyramidâ. This basically put into words what I had discovered. The simpler the test, the easier it was to maintain (you can find many pictures of the test pyramid that talk about cost and how it goes up as you go up the pyramid). Additionally, if you had a system level test and there were a dozen reasons it could fail then it was time consuming to figure out why. But if you had unit level, integration level, contract level, etc. then you could spread out where the failures occurred.
If you test things in sequence, then you KNOW defects will be found at the unit level before they make it to the higher levels. Defects will be caught at the integration level before they make it to the higher levels. But if you start testing things in parallel or not in a sequential order then you, in my opinion, defeat the purpose of the test pyramid.
So to me, we automate so we can fail fast. The test pyramid is more about making things easier to maintain and debug.