I read a blog post from Alister Scott about his thoughts on Karate for test automation “Karate: Test Automation Made Simple?”
The post got me wondering if other people had experiences with Karate?
Have you used it for web browser tests? Or for API tests? Or both?
I’m also interested in the size of the organisation you used it in after reading the closing summary from Alister:
I am not sure how well it would scale to a med-large software project considering things like configuration and reusability seem difficult to implement in plain language gherkin files.
I’ve seen a number of teams implement essentially the same concept using Cucumber + a set of step definitions that bind to HTTP elements, and I think it’s fundamentally a misguided notion.
Gherkin is a very costly abstraction to include in your testing stack, and its value is only realised when it enables non-technical project stakeholders to participate in feature definition.
As soon as you start mixing HTTP and HTML/CSS primitives into the text of your feature specs you’re rendering them inaccessible to 99% of their non-technical audience and thus voiding their benefit, while sticking yourself with a language that doesn’t support refactoring, abstraction, (non-trivial) data structures etc.
I agree with @tomakehurst. We already use Cucumber for a few systems, it’s been working well and I like it.
One team wanted to use Karate instead and I could not see the advantages.
I’m sure it’s a nice framework, but the main purpose of ATDD/BDD is as communication in my experience. That we get automated tests is a nice addition.
Karate tests are too technical for the business side to understand them, so you lose the communication aspect of it. And there are so many other frameworks that are faster or more connected to the developer environments.
I would rather put the time into making sure that the tests in whatever other framwork that we use have good names and description. Making it easier to understand what we do and do not test.
This is a common misconception which is addressed in this article: https://hackernoon.com/yes-karate-is-not-true-bdd-698bf4a9be39 - Karate is not at all meant for a non-technical audience.