Cucumber / Gherkin - Preventing feature file proliferation

(rob) #1

Hi I’m currently using a cucumber implementation to test mobile apps on Android and iOS. I’m using a Git repo of gherkin scenarios tagged to identify the relevant apps and platforms (e.g. app1, app2, ios, android) where all the procedural elements of the test are abstracted out into the step definitions (written in Kotlin and SWIFT). I’m doing it this way to try and prevent unwanted proliferation of Gherkin scenarios where we might have multiple identical gherkin scenarios for different apps e.g. login fo app 1 and login for app 2

So if I give a pseudo code example

GENERIC GHERKIN SCENARIO

Feature: App log in

Scenario: User can log in with valid credentials
Given the user is on the log in screen
When the user attempts to log in with valid credentials
Then the user can log in

Note that there’s no detail in here about hard coded credentials, which fields credentials should go in or which buttons should get tapped etc . It’s a behavioural test and currently holds true for all our apps where there is a log facility.

Step definitions for App1

step(“the user is on the log in screen”) {
Launch app
Assert user is on log in screen
}

step(“the user attempts to log in with valid credentials”) {
Fill in these fields to log into App1
Tap login button
}

step(“the user can log in”) {
Assert user is now on correct post login landing page
}

Step definitions for App2

step(“the user is on the log in screen”) {
Launch app
Assert user is on log in screen
}

step(“the user attempts to log in with valid credentials”) {
Fill a whole set of different fields to log into App2
Tap login button
}

step(“the user can log in”) {
Assert user is now on correct post login landing page
}

My questions are

  1. ‘Does this look like a sensible approach to people’

  2. ‘Can people see any downsides to the way I’m doing it?’ One immediate downside seems to be that by having all the detail in the step definition it might make things awkward if I ever wanted to use the Gherkins as a guide for manual testers.

  3. How have others approached the same issue of preventing feature file proliferation?

Thanks
Rob

(Constantin Weiss) #2

Hi Rob,

We are not using Cucumber in our Team but maybe I can give you a few hints.

  1. I think your Gherkin is fine as it helps understand the user story and adds acceptance criteria.
    For me the definition of acceptance criteria through gherkin is a key feature. Otherwise it is hard to track test coverage and testability. Especially in Agile Teams Gherkin and BDD is helpful as User Story’s tend to have too much room for interpretation.

  2. You are at the moment just looking on the happy path, i think best practice is covering exception handling as well.
    You could use Scenario Outlines to define the Test Environments to handle duplicated Scenarios.

  3. That is a good question. I normally catch this during writing the scripts for the test.
    In your example I would write one Test for login with a paramterized target environment.
    Data driven parts (data tables) or scenario outlines are the only way to handle this in Gherkin itself, as far as i know.

Cheers,
Constantin

(rob) #3

Hi Constantin. Thanks for the feedback. We’ve been working with this approach for a few weeks now and it seems to be going ok (we are including -ve scenarios too but the example above was just intended to be as simple as possible for illustration). We ran into one issue which wasn’t really gherkin related but was that as the Android and iOS teams delivered features at different rates test suites would fail because there would be step definitions for a gherkin scenario in say Kotlin but not SWIFT - we got round this by just tagging scenarios in the repo so that they’d only be executed when each team was ready.

As for manual level testing…well people either need to look at the step definitions or external docs as well as the gherkin if they need actual ‘click here / enter this’ type detail. Not an issue for us as we’re primarily using this approach to support automated executable documentation. Cheers, Rob

(Paul) #4

I’m in a similar scenario myself.

We are trying to avoid duplicating feature files for different tests types, e.g. Manual Test, API, Mobile Browser (Appium) and Desktop Browser (Selenium).

The plan is to create a unified Scenarios held in a central repository.

The write the separate step definition for API, Desktop and Mobile based on the definition in the central repository.

It’s very early days at the moment as we are just writing the feature files at the moment. But I’m aware this could be a very brittle process because if somebody updates existing feature files, it has the potential of breaking API, Desktop and Mobile automation.

If anybody has been there and done this and has the solution I’m all ears :slight_smile:

The last thing I want to do is create separate feature files for each test type.

(rob) #5

If either of you are interested we could do a zoom session at some point and I can show you what we’ve done. We’re still a work in progress but some feedback would be great. I’m sure I can show the tests without getting into anything commercially sensitive.

(rob) #6

Eventually this gets too tiring to type!

(Paul) #7

Hi Rob, yes that would be really helpful please. Thank you for the offer.

(rob) #8

Ok cool. I’ll set something up for after Easter.

(Paul) #9

cool, thank you very much :+1: