We have been using Specflow & Selenium now for about 1.5 years and have noticed that it seems a bit over complicated at times.
I understand the idea is to unify the language used between business, designs, dev and test…but I don’t know of anyone who has successfully got the business to talk in this language, making the whole requirements into gherkin language just another task to complete. We also seem to have the occasional issue with getting it to work on everyone’s machines.
What are your experiences with Specflow? Has there been any real WOW moments, or could it all be achieved far more easily just directly writing tests with descriptive names and utilising the POM style to help with readability?
We used SpecFlow for evaluating APIs. When we started, our analyst had not written Gherkin for APIs but we agreed to experiment. Early in the project, we discovered some duplication of SpecFlow automation with unit test automation. We met with our development team and agreed on a boundary. The SpecFlow automation would evaluate behavior, that is, everything that could be explored by executing a request and inspecting the response. The unit tests evaluated everything beneath that such as database interaction, request preparation, and the like. Mocking was required and encouraged.
Together, we created a suite of automation that provided accurate feedback quickly on additions and changes to code.
In my opinion, we have a suite of tests that can demonstrate business behaviors at any time. I think this is beneficial for projects that may update the API, or business team members who may research existing behaviors.
I’m unfamiliar with the POM style. Would you elaborate?
my experience is not with SpecFlow but rather with Robot Framework in a couple of companies. The QA team used the tool to write some functional tests using gherkin style. The promise of “language that would re-unite business, designs, dev and test” was never achieved. The tests always remained QA ownership. Though we had Dev and Doc looking at our tests from time to time (essentially to check what the product is supposed to do!). All in all it was a success anyway because those functional tests harness were a super useful and quick feedback on the quality of the product, but not the universal langage that some expected.
Thanks Joe, that’s a really good point, it does help document the existing behaviours! I’ll use this for any future on-boarding!
POM stands for Page Object Model. In essence you create a class for each page of your application, this class models the buttons, links, input boxes etc.
(In C# with Selenium it can look like this:
private static IWebElement LocationInputBox => Driver.FindElement(By.Id(“location”));
)
It also provides public methods for the behaviour.
By using the methods in your tests you can make your tests super simple to write as well as separating the actual selenium code from the test too.
We have started using POM as it creates an easy to maintain model of your application which allows testers to quickly get up to speed and start writing tests
POM features in a Plural Sight course called Automated Testing: End to End. This course calls it a Physical Object Model, however I’ve not seen it called that anywhere else (yet!).
Another good guide (using Selenium) can be found here
Thanks Paul! I’ve also seen this referred to as the DOM (Document Object Model). In the end, the model represents the page content allowing a test engineer to manipulate or inspect the page at a code level.
I worked with Web Driver a while back and found it effective for driving a website for the purpose of creating data on the back end. Most implementations I’ve seen of web application automation intended for assisting testers require a bit of maintenance so I’ve grown skeptical to their benefit. I found this especially true for end-to-end tests (e.g., log in, perform a multi-page transaction, and check results).
Where the application design includes APIs, I favor unit tests and behavioral tests much like we discussed above. In that manner, testing at the UI level becomes an evaluation of placing the data at the right spot on the page rather than exercising business behavior through the UI. In my opinion, the separation allows better coverage at both levels.
