Oh. This is hard. With agile methods, the team has much more options to influence the solution. So don’t try to just replace one element of the old process, but thing out of the box. The things go wrong when the team just “replaces” the “XYZ-tests” of the old process with TDD and then realize that it is not conveniently fitting.
Not a very detailed advice, sorry.
A good preparation for the scenario-based story implementation is to have a good list of expectations in a from of business rules or acceptance criteria, but with the openness to change or extend them based on the discussions with the team.
From the solution side, I think having a good design concept is a good starting point. I definitely would not go into detailed solution design decisions upfront.
We will release a new increment of our beta book (Formulation, BDD Books – Formulation by Seb Rose et al. [PDF/iPad/Kindle]) this week. In the new Chapter 3 we also talk about this.
Generally the best approach is to see whether the problem you try do describe is calculation-like (data-driven). For those kind of problems, a table with the input-output pairs is an efficient way for discussion and understanding. For these problems converting these data-variation tables to Scenario Outlines is natural and it documents the results in an clear way.
Other problems are more about the workflow or the behavior. For those cases, using Scenario Outline and test the things with a couple of more data points (just because we can) is usually an overkill.
Yes, this is definitely an interesting area that will develop further. I haven’t seen much concrete application that would really combine these, but I know about a few people who try.
I was playing with combining property-based testing with BDD once (see Property-based BDD Examples with SpecFlow and FsCheck - Gáspár Nagy on software).
Jenny Martin has a very good concept that is called the Data Personas (Data Personas | @Jennyjmar). It is basically the same UX “persona” concept applied to any other data as well. I think stealing some techniques from UX how they manage and document their personas would be also beneficial for managing test data.
Wow! Done!
Thanks everyone for joining. If you have any further questions drop me a mail (gaspar at specsolutions.eu) or find me at the London Testers Gathering Workshops in June!
Have a good evening!
Thank you! I like the idea of intention versus mechanics.
Yeah. We “invented” the intention/mechanics pair with Seb in the scope of the Formulation book, because the declarative/imperative terms are a bit too techie and therefore not so broadly understandable.
I seem to have missed this question yesterday…
The automated tests/scenarios should be focused, regardless whether they are automated through the GUI or not (see more on this at the question about manual/automated tests above). In many cases this can be achieved with a scenario that has not more than 5 steps. (This does not necessarily mean that you have only 5 UI actions, because one step can trigger multiple UI actions as well.) With UI tests the key challenge whether you can automate the preconditions (the “Given” steps in Gherkin) in a robust way, because the most of the instability and slowness is actually caused by these steps. Making them robust might mean that you bypass the UI (e.g. setup the necessary data through the REST API or programming API of the application) or make testing backdoors so that the test can jump directly to the UI to be tested (e.g. you should not need to automate clicking through a wizard just to be able to test the last page).
In general, GUI tests should only be used for limited number of tests. The majority of the testing should be and can much better be performed with API-level integration tests or even unit tests.