Automatable acceptance criteria

I work for a company where we have moved from waterfall to agile but still doing manual testing. We have web applications and in house desktop applications where we also have a lot of regulatory governance resulting in changes we have to make alongside the enhancements we want to make. We moved from working out of excel in the past to now working in dev ops and writing user stories for all our changes in an agile fashion which improved the quality of the details we were getting. We are now looking to move to automate as much of the UI testing as we can and what makes viable sense in terms of re-use but feel we need to look at the user stories acceptance criteria. I have looked around online and behaviour driven development seems to be an obvious starting point, “GIVEN, WHEN, THEN” but im still finding it hard to truly write some of our stories in this fashion as they can be quite technical with a lot of variables around what the change may be. All the literature i have seen so far tends to be the most basic of examples like making a cup of coffee which just doesnt translate back to what most of our backlog items look like. Can anyone help me with a good point to start at, what did you do on your journey to automation if you have been through this phase with a company? Any examples someone could share with me? Is there a different/better option to GIVEN/WHEN/THEN? thanks in advance

2 Likes

Your “manual testing” triggers me hard, I try to stay calm. (Edit: see the P.S. for details)

The automation of actions and checks is just one tool out of many others. You need to do it appropriate.
If you want to automate often repeated steps, that’s fine. “Test actions” which is are kind of robotic.
But don’t expect to get rid of investigation of bugs and exploration your product in general.

My of my key insights within 10+ years of automation:
You can really start (or fully finish) automation only once the software, the actions you want to automate, are bug free.
Otherwise you will constantly stumble over bugs during the automation. Doing the testing while you automate.
Making temporary workarounds isn’t fun. Especially getting reminded once you have to remove them so the original actions and check take place.

Another thing: Specifically developing, and even more maintaining, UI automation is much effort.
Imo you should here really carefully look for the trade of.
Not all UI automation is it worth.
The key issue here: UI is an human2machine interfach which we try to use by machine. machineLikeHuman2human.
Humans have no big problem with invisible changes in the DOM, but your computer can.
It would be good to have support from the developers by increasing the testability of your UI.

It think of automation in different ways:

  • Providing data for further testing. E.g. having done long running actions automated so I can continue after that.
  • Doing some superficial smoke checks but not to in-depth. See trade-of above
  • Reading out data for easier assessment by a human. E.g. gathering data from the DB into an spreadsheet

Automation is basically development. It’s demanding.
And with development in testing you can do much more than automation.
I create for our team every once in while small tools for different things.

About Given/When/Then:
This imo works mostly on a more unit level where you can separate things easily.
Imaging a typically purchasing at an online shop. You want to test specific details in the checkout. Can you imagine how much Given and When you need to come to the checkout? It can become a lot.
And also the Then. I seldom check only one thing but multiples.
My suggestion: Take a normal xUnit framework, give meaningful names to methods, classes and packages. Maybe add some comments.

One more thing about automation:
Don’t just think of it running unattended at your CI server (Jenkins, TFS?) at the nightly.
But also think at use it for your actual testing. Automate what could be easily done and fill in the gaps where the computer troubles.
Attended “Semi”-automation can speed you up at your local testing. And you can make easily adaptions to it.
Often the IDE of my automation is one of my important test tools while I test something.

I hope you find something useful in this. :slight_smile:

P.S. I feel triggered because I see questions like this coming up in a frequent manner. I could copy&paste my answers often.
P.P.S. There are no wrong questions. Just my frustration about the general state in the software industry and the widespread unwillingness to educate people.

6 Likes

As has been identified by @sebastian_solidwork there are lots of pitfalls in doing this.

I agree that lots of examples on the web are very simplistic. For some that are more detailed I found the book BDD in action by John Ferguson Smart to be useful.

I found this 15 minute talk quite interesting

There is a link in the description to another talk by John Ferguson Smart that I also found interesting.

2 Likes

It was nearly 20 years ago when I started working with teams making the journey from waterfall to agile.

In the waterfall world I saw a lot of testing at the end, it had a lot of planning and design but likely one of the most distinguishing features was that had a verification focus so they has expected results and tests would pass or fail. In waterfall you also get a lot of UI layer automation covering the regression risk.

Moving to more agile models, for me the change was not primarily about automation although that did adapt aswell. With the team more collaborative and developers often taking on more automation it became more optimal to rebalance coverage at different levels in the stack. Given, when, then is only useful as far as the whole team finds it useful, if someone is solo on automation and nobody shows much interest in understanding their coverage its unlikely to be seen of as value to the person automating at all. You will have to break down your tests to be less complex and this may be a good thing as you need to consider were your old tests over complex?

The primary change though I saw from a testing aspect though was more of a shift from a verification focused model to a discovery focused model, that is a massive mindset change from a testing value perspective. I see a lot of teams not getting that change and bring with them the verification focus to Agile, its not a bad thing or worse than what they had in waterfall but it does miss a lot of opportunities to do better testing.

I flag this because you mention ‘still doing’ so potentially looking at testing in the same way when often you can revisit what testing is all about and that may also impact how you look at automation.

2 Likes

Thanks for this interesting information!

1 Like

If you are looking to hire a dedicated software team, there are several things you should consider before making the hire. First, it is important to have clear requirements, and make sure you document them. You can do this by documenting the requirements and by providing a few basic details about the company. This will help the dedicated team get a better sense of the company culture. Interviewing a dedicated software team includes various stages. The vendor team will access a short-listed team of candidates through an oral and technical round. Then, the client interview shortlisted candidates https://mlsdev.com/services/ui-ux-design. During the interview process, the company’s executives will evaluate their communication skills and interpersonal skills.