A Situation that my manager has posed to me about e2e/UI testing

🙋 Questions
,
1

So my boss comes to me and notes that a new product (an MVP in production) currently has no UI/E2E testing in place and that I should add to my list of stuff to implement such a thing.

Here are the factors.

  • My boss is a product owner. My boss is familiar with the concepts of testing but not so much the deeper technical aspects. And really, she doesnt need to be. She wants the product tested and Users not encountering defects.

  • I am “The QA guy” and Im a new hire. So Im still figuring many things out. Im aware of the product and have UAT one small story on it. I am more familiar with the flagship product. The newer product likely follows the same tech stack. (I only got access to the repo yesterday)

  • Thus far, dev engineers have been responsible for the quality of their code and implement (a good swath, I must say) automated unit and integration tests

  • On the flagship product the E2E and UI validation is done by a few Cypress + Percy tests.

  • When I asked about what might be desired the response was “Whatever you want. There is nothing in place so you can do whatever you want” Which is a unique opportunity in my experience

So here is what I am weighing.

  • A whole new E2E test framework! Something offering the newest hotness that will make it easiest for the engineers to add and update new tests (I simply cant do it all in a timely fashion). Its always fun to dive into something new and amazing. On the downside - I dont know what that might be. Playwright seems to be on the boil lately. Ive seen a bunch of other frameworks getting some sort of buzz here. I dont know what infrastructure might be needed for the new hotness in order to integrate it into CI, report issues, etc

Or…

  • Just stick with Cypress. Even though the engineer teams are sharply divided, there is familiarity within the company with that framework. so it is a known quantity. I think I would like to extend it into TraceTest, however. Then we can gain some even more valuable tests out of cypress. On the downside, I dont like that the current implementation of the automation Cypress Cloud, only runs Electron (but that might be solved by moving to browserstack). I dont like that webkit isnt well supported (but thats a minor nitpick I can live without)

My instinct is to stick with what is there. No matter what is selected, it will have drawbacks and warts to overcome for our use so why not stick with the ugly we know?

Im interested to hear any thoughts or frameworks I might not be considering

2 Likes
2

Given the situation you’re in, sticking with Cypress seems like a practical move, especially since you already are familiar with it and company has a solid understanding of it.
Expanding Cypress into TraceTest could add valuable tests while keeping things within the team’s comfort zone.

However, if you are feeling adventurous and want to explore new possibilities like Playwright, it could be an exciting opportunity to bring in some cutting-edge technology. Just make sure to consider how it will integrate with your current setup.

Whatever decision you make, the most important thing is to ensure that your testing strategy remains effective and efficient, so the product meets the high standards expected by your user.

3 Likes
3

Looks like you’re putting tools ahead of anything else.
I’d get back a few steps and identify what would be the most efficient way of spending your time to achieve some level of automation at the UI level that’s reasonably helpful for you, the PM, and the developers.
I went through multiple frameworks, but in the end, the main effort didn’t seem to be the coding. I found important:

  • stability of the product you automate against
  • consistency across similar controls/functions for the product
  • data creation, edit, destroy - through APIs, database, complete environment refresh, predefined and configured;
  • product state availability
  • availability of use-cases/scenarios which could be made reproducible for days and months in the future;
  • variability of data or action which should increase the chances of finding issues;
  • consistent maintenance/update and resources available to do so;

For the tooling, I simply asked the developers what they thought about the current approach/toolkit, and what would they prefer. In half an hour we had a path already selected.
I would never assume that the UI automation would have any other contributors than the one that implemented it initially. Chances are if you leave, it will get trashed soon.

2 Likes
4

Founder of Tracetest here - would love to see you extend your E2E tests with Tracetest. We recently enabled the observability enabled testing to also work with Playwright - so either will work well. Here is a blog with a video discussing it and links to the docs: The Lord of Playwright: The Two Traces

Let me know if you need any assistance and we will help.
Ken

1 Like