A great conversation starter came up on MoT Slack recently:
I actually would like someone who uses Cucumber or JBehave to convince me, because it’s so prevalent, for some reason
I wanted to reply on Slack but since this is a topic that comes up frequently, I felt a Club post was more suitable:
I think one of the issues is that Outside-in-Development (ATDD and TDD combined) gets conflated as BDD when really Outside-in-Dev is a part of BDD. Typically when we discuss Cucumber we are referring to Outside-in-Development and not the whole of BDD. With that in mind, if you want to define the value of Cucumber, you have to go back to risk, specifically the risk of building something the business doesn’t want. The point of Outside-in-Dev is to guide developers with automated examples by automating them to fail and designing production code to make them pass. Cucumber is used as a scaffold in which the developers can build a product. An analogy I like to use is: automated examples are like bumpers on a bowling alley. They stop the developer from bowling a gutter ball but leave space for them to design code that is a strike or a single pin.
The risk of incorrect delivery and it’s mitigation is very different from automated regression checking though. Automated regression checking is dealing with the risks of changes in a product and how we deal with said changes. Cucumber, or more specifically Gherkin, isn’t a great tool to use for detecting change across a complex system because GWT is designed to focus on the product as a whole. This typically results in the usual pattern of testers trying to automate numerous flows through the system from a high-level system perspective, which has inherent problems around brittleness, maintenance, feedback, etc.
Sure you get readable tests, but if they are tests that are designed to exhaustively (or at least attempt to) test an application. Then they cease to be documentation and are no better than test cases. Which in my opinion offer better readability than Gherkin (caveat: in the context of testing!!!)
So in summary. Cucumber, or any other Gherkin based tool, is used for mitigating risks around developers delivering the wrong thing and when you have a mature team that can develop around that approach, it’s great! If that is the risk you are mitigating, then go nuts! However, if you are focused on automated regression checking. Then I would encourage a team to go a bit deeper into their product, understand it better, leverage interfaces nearest to risks that they are trying to test and basically avoid using Gherkin tools to automate everything.
P.S. Yes I know you can still use Cucumber for testing regardless and yes it may work for you. But I would challenge the naysayers to find a better way to automate. Just because a tool isn’t the right tool for a job doesn’t mean it won’t work. It simply means the work is much harder!