Yesterday I had a demo of Cypress. I was very impressed, and had much fun playing around with it. Cypress claims to solve the problems of brittle and flaky test automation.
I know it is still in closed beta, so maybe not many people have tried it yet. But who has and can tell me a bit more about the benefits and drawbacks in real-life situations?
(Of course I’m willing to share my (very limited) experience too if people are interested/curious.)
I haven’t used Cypress yet. Looks like it is a really slick interface. Signed up for pre-beta access. I would be curious to see how/if Cypress can manage test data. This was one of our biggest problems we faced when implementing automation. One notable difference is Cypress looks like a framework that runs your tests while Selenium is more of an API that allows you to drive the website. You still need a testing framework to run the test and other tools to make selenium easy to use.
You can run the tests either from the Cypress desktop app or include them in your build process (took us 30 minutes to set up).
We set up a couple of tests now, and will monitor in the upcoming weeks how they help or hinder us.
I myself have hardly any programming experience, and even I got this up and running. There are many examples in their documentation, that you can just copy-paste and modify. So I wouldn’t let lack of JavaScript knowledge stand in your way.
Here is an example of one of my tests (surely you can go a lot deeper and I intend to do so later):
This is a fixed format, you could happily work with this by just replacing the lines starting with ‘cy.’ and not knowing anything about how to write functions in JavaScript. All the functions I used were copy-pasted from their documentation.
The hardest thing for me was to get familiar with CSS-selectors.
Hi there! I switched projects, and my current project has limited use for end-to-end testing (since it is a minor application).
What I liked about it (from small proof of concepts):
Transparency: you can play back the test, see where it failed (actually see, so including the screen it is on, where it tried to click, and why the clicking failed (for example: ‘found multiple elements with class xx so I don’t know which one to click’)
Relative easy of use: I was new to both JavaScript selectors and Mocha, so I had a long learning curve ahead. But I could get up&running quite fast
Integrated well with our CI
Runs very fast.
Downsides I experienced:
I had frequent crashes of Cypress itself. So while developing the tests, not while running them. I mentioned this on the Cypress-board, but it has not been addressed as fasr as I know.
No cross-browser testing. IMHO this is not a big deal for our product, but with Cypress you can only test on Chrome and Elektron, not on IE, FF, etc. For some this might be a dealbreaker.
I would highly recommend you to do your experiments. Since it is low-cost to get started, I would see no reason not to.
I have been using it for one of my projects lately, and I don’t think I like it lol
They are anti-page object model, and they encourage some best practices that are not feasible in the real world. I’ve been trying it out anyway but it’s been kind of frustrating going through their documentation for some things, like simply how to identify elements. Maybe it’s because I’m not as familiar with JS that it’s causing me more trouble than others…
I have experience using it and i have no coding knowledge before i started, i set it up myself and was quickly on my way to writing tests. It has opened up so many doors for me and expanded my knowledge massively. The documentation is easily followed and the suggested search functionality gets you the help you need in seconds. The interface is cool being able to watch exactly what the tests you have written are doing and at exactly which point your going wrong so your not sieving through lines upon lines of code to find the issue. However i haven’t used selenium before so i’m unable to comment on a comparison but it is something i’m currently looking into. I couldn’t recommend cypress enough though.
My experience with Cypress is mostly positive. I made some basic flows for what I need and also integrated into Jenkins pipelines, BUT I got some downsides as well.
Just recently, Dev team decided to implement Hammer.js into the project, because they wanted to replace click listeners for tap. (app suppose to run with touch screen)
That disabled my tests, because cypress doesn’t know tap. I mean… it still clicking on certain element, but the app is doing nothing, because it listens on tap only. Currently I’m looking for solution.
I came across this article which compares Cypress and Selenium Webdriver. Maybe this is helpful. Personally, I think I’ll stick with Selenium Webdriver for now.