srj
(Sreejit P)
8 May 2021 14:19
1
Selenium webdriver, open source UI tools VS testproject kind of cloud platforms
When I started working for a new company, one of the first things I noticed was that they have a tendency to do UI automation with scriptless tools available on the market. The actual application to be tested is with an Angular front end (Nothing much complications I foresee on the UI except some pdf validations). Its more of a web based application with at this point no device support required(Mobile etc)
I worked with java spring boot based micro service system before joining here. We used to split the tests into three layers: Unit testing, API layer and the final collection of end-to-end scenarios on the front end along with the automated code quality scanners in the pipleine. Based on the context, we would probably choose an open source tool from this list
Selenium Webdriver with Java or some other client binding
Microsoft Playwright
Google puppeteer
Nightwatch JS and so on.
Here they are, however, attempting to promote testproject. Except the cloud solutions sauce labs and browser stack, which allows us to easily test multi-combination devices for the scripts we build in our standards, I am not at all a fan of these types of recording and playback platforms. These kinds of platforms usually come at a cost. However, testproject.io has a FREE model (which I assume is not open source) and Tricentis tells their own stories why they keep this free. The selling point of my superiors is that the people with non programming experience can use this, which I am not at all convinced. In 2021 I would definitely suggest to increase the people skill and not dive back 2010 thoughts
Is it a good idea to select these kinds of platforms as I strongly feel this recording and playback will surely become a headache going forward. Could you please share your thoughts on this?
I had raised the same in sqa.stackexchange.com where somebody suggested protractor. The reason I left Protractor is that I’m not sure how much production and maintenance is currently taking place. If I search protractor’s github, the most recent commit (the most recent master branch commit is on May 14, 2020) is at least an year old. Even though the protractor is from the angular squad, I’m not sure how much people are maintaining protractor after Pupeteer arrived on the scene. Also, I’m not sure whether Protractor is still using older selenium webdriver versions or whether they have upgraded to the new w3c webdriver protocol under the hood.
Again my question is more focused on testproject kind of recording and play back tools
1 Like
mirza
(Mirza Sisic)
10 May 2021 08:00
2
Protractor is officially being retired, so no sense in using it anymore:
opened 08:44PM - 05 Apr 21 UTC
closed 06:06PM - 20 Apr 21 UTC
# TLDR
The Angular team plans to end development of Protractor at the end of … 2022 (in conjunction with Angular v15).
Why?
Protractor was created in [2013](https://github.com/angular/protractor/commit/0595652c2cf936419a08b82a866cc5c0915b8a6c) when WebDriver APIs were not yet a [standard](https://www.w3.org/TR/webdriver/) and end-to-end (e2e) tests were hard to write due to lack of support for `async` / `await`. To solve this problem, Protractor wraps [selenium-webdriver][1] and abstracted asynchronous operations from developers with the use of [Control Flow][2].
Since then, the JavaScript standard and ecosystem advanced considerably, providing modern syntax and much better development tools. Nonetheless, Protractor is not able to leverage such technology without forcing users to rewrite their tests. Meanwhile, robust alternatives have emerged in the web testing space. Developers will see more benefits from adopting a more modern testing tool than from updating to a breaking version of Protractor which does not provide additional functionality or developer ergonomic improvements.
We would like to hear from the community on
- the deprecation timeline
- what we can do to provide reliable integration with third-party solutions
- how users can transition by following migration guidelines
- additional concerns that would ensure a smooth transition
This RFC will close on Friday April 16, 2021.
---------------------------------------------------------------------------------------------------------------------------
# State of Protractor
The Angular team created Protractor in [2013](https://github.com/angular/protractor/commit/0595652c2cf936419a08b82a866cc5c0915b8a6c) when WebDriver APIs were not yet a [standard](https://www.w3.org/TR/webdriver/) and end-to-end (e2e) tests were challenging to set up. The proliferation of callback patterns in JavaScript back then made asynchronous operations difficult to write and manage.
Protractor, echoing the approach taken by the underlying [`selenium-webdriver`](https://github.com/SeleniumHQ/selenium/tree/trunk/javascript/node/selenium-webdriver), solved this problem by managing promises via [Control Flow][2]. Control Flow makes asynchronous calls appear synchronous, thereby avoids the use of Promises entirely.
This strategy worked well for some time, but as the JavaScript standards and toolings evolve, Protractor regressed. Starting from ES2017, `async` / `await` makes handling Promises significantly easier because they no longer have to be chained. However, `async` / `await` is fundamentally incompatible with Control Flow, and support for Control Flow is [dropped](https://github.com/SeleniumHQ/selenium/issues/2969) from `selenium-webdriver` [v4.0](https://github.com/SeleniumHQ/selenium/releases) completely. Protractor, being dependent on `selenium-webdriver`, is not able to upgrade to the new version without introducing a huge breaking change and forcing users to do a migration for all their tests.
Besides dropping support for Control Flow, making Protractor compatible with ES2017 involves a significant amount of work to overhaul its implementation. Legacy dependencies such as jasminewd2 and Q promise library ought to be removed, which will bring about further breaking changes.
Since Protractor was initially designed to support AngularJS, many of its features like locators and mock modules are specific to AngularJS. These features only work in AngularJS and not Angular. They will no longer be relevant once development on AngularJS ceases by [December 31, 2021](https://docs.angularjs.org/misc/version-support-status).
Removing Control Flow and dropping AngularJS-specific features would make Protractor essentially just a wrapper around selenium-webdriver that provides no additional functionality.
# Current Landscape for Web Testing
Many testing solutions have appeared since the creation of Protractor, and they offer better stability and much improved support for features like cross-browsers testing. Some even eschew the WebDriver standards completely in favor of using the DevTools protocols directly.
In order to understand the adoption trend of these alternatives in the Angular community, we conducted a survey on e2e testing in January 2021. We received close to 1000 responses and got a lot of feedback from the community. A clear signal that we received is that **there is no one-size-fits-all solution for all Angular projects** out there. Fewer than 20% of the respondents use Protractor in their project. Some users prioritize the cross-browsers support that the WebDriver standards guarantee, whereas other users prefer the stability and flexibility that DevTools protocol offers. A similar survey focused entirely on enterprise customers yielded comparable results.
<img width="1011" alt="Screen Shot 2021-03-24 at 6 31 33 PM" src="https://user-images.githubusercontent.com/2941178/113600541-0480ad00-95f5-11eb-8232-69e049638acd.png">
# Moving Forward
Deprecating Protractor is not a decision that is taken lightly by the team. Inside Google, we support thousands of projects and hundreds of thousands of test targets that depend on Protractor. After evaluating the costs and benefits of updating Protractor vs migration effort imposed on users, we conclude that the best strategy to set users up for success in the long term is to encourage migration to a more modern and framework-agnostic testing platform. The reasoning is as follows:
Removing Control Flow from user code is a gigantic effort because it forces users to update all their tests. Google went through this effort, and it took a major infrastructure team half of their time in 2018. Even then, they could only migrate tests written in TypeScript due to the availability of more robust tooling for source code analysis and transformation.
After going through such a massive endeavour, we realize the “new” Protractor is no better than other existing solutions. The only feature that differentiates Protractor from `selenium-webdriver` at this point is the ability to automatically wait for the application under test to become stable ([`waitForAngular`](https://github.com/angular/protractor/blob/df3dd6400e3a4792ff9f7a58849d2a5eed6e1483/lib/browser.ts#L620) feature in Protractor).
Although `waitForAngular` is useful, it strongly couples the testing platform to the Angular framework. Some teams at Google have found that solutions that do not require knowledge of Angular can perform the tests equally well by using a robust retry strategy. We believe this is how e2e testing should be done going forward, and projects in Google are already converging towards a testing platform that is WebDriver compliant and framework agnostic. This allows our web test team to maintain a single solution for all web applications.
Externally, there are many excellent alternatives available to the open source community, such as:
1. [Cypress](https://www.cypress.io/)
1. [PlayWright](https://playwright.dev/)
1. [Puppeteer](https://pptr.dev/)
1. [Selenium-webdriver](https://www.selenium.dev/)
1. [TestCafe](https://devexpress.github.io/testcafe/)
1. [WebdriverIO](https://webdriver.io/)
(The list is sorted in alphabetical order and is non-exhaustive.)
# Deprecation
The most important focus for the Angular team is to ensure a smooth transition for Protractor users. We have been engaging with different vendors to ensure
- there is sufficient documentation / tooling to help users migrate to other platforms
- third-party platforms integrate well with Angular CLI, so that CLI configurations are automatically handled by vendor-specific builders
So far, the Angular team has reached out to the teams at Cypress and WebdriverIO to achieve the goals above. We will provide regular updates as more resources become available. If there is a particular platform that you’d like us to reach out to, please let us know in the comments below.
Our proposed deprecation timeline is as follows:
- Angular version 12 (May 2021)
- Announcement of deprecation
- Angular CLI will not include Protractor for new projects
- Show warning when `@angular-devkit/build-angular:protractor` is used
- Only PRs that address security issues and fixes for browser releases that break Protractor tests will be merged
- Angular version 15 (end of 2022)
- End of development on Protractor
_No decision is final until we assess all the feedback from this RFC._
# Extended LTS
For users who require extended long-term support after the end of life of Protractor, we suggest reaching out to our external partners:
- [flare.build](https://flare.build)
- [xlts.dev](https://xlts.dev)
If your team / company provides similar services, please let us know!
# FAQs
1. What about [Component Harnesses](https://material.angular.io/cdk/test-harnesses/overview)?
Angular CDK's component harnesses include a `ProtractorHarnessEnvironment` for using harnesses in Protractor. This environment will be deprecated and replaced with a new environment backed by `selenium-webdriver`. The harness APIs themselves will be **unchanged**.
1. How will Angular CLI handle `ng e2e` for new projects starting from version 12?
The CLI will not include a default e2e builder for new projects, and users will decide which third-party builder to install. This is because picking an e2e platform really depends on the requirements of the project and there is no clear best fit for all Angular projects.
1. In version 15, would all Protractor tests break?
The Protractor builder will continue to work until version 15. After that, we plan to remove the Protractor builder (`@angular-devkit/build-angular:protractor`), which means `ng e2e` will no longer run Protractor. However, we are open to revising the timeline based on user feedback. Please let us know in the comments below.
1. Protractor is useful because it automatically waits for the application to become stable. How can I use that in other platforms?
Under the hood, Protractor uses `Testability` provided by `@angular/core` to check if the app is stable. This feature can be integrated into other platforms in a straightforward way. See the [example](https://github.com/kyliau/angular-wait/blob/main/server/server.ts) for `selenium-webdriver`.
1. Which e2e framework is the lowest effort and cost to migrate to from Protractor?
In terms of APIs, `selenium-webdriver` is most similar to Protractor because it is used under the hood. If your tests still use Control Flow, you’ll have to remove it first by adding `async` / `await` to the Protractor calls. After that, you can replace the Protractor methods with those in `selenium-webdriver`. Although they are not 1:1 replacements, they are very close.
1. Where will e2e migration strategies be documented?
We are consolidating a list of migration strategies doc written by vendors and community members. See [Links](#links) section below.
1. What other packages will be deprecated along with Protractor at the end of 2022?
- [protractor-cookbook](https://github.com/angular/protractor-cookbook)
- [jasminewd](https://github.com/angular/jasminewd)
- [webdriver-js-extender](https://github.com/angular/webdriver-js-extender)
- [blocking-proxy](https://github.com/angular/blocking-proxy)
- [webdriver-manager](https://github.com/angular/webdriver-manager)
1. My team just did a migration to disable Control Flow. What are our options?
It is possible to create a thin wrapper around `selenium-webdriver` to provide APIs that are compatible with Protractor's. This is an idea that needs further exploration. Please let us know in the comments below if such tool is useful to you.
# Open questions
1. Should the Angular team provide a different deprecation timeline?
1. In addition to migration docs and CLI integration, how can the Angular team support Protractor users in the transition to other platforms?
1. What are the alternatives to deprecating Protractor?
# Links
This section is a work in progress. We will continue to consolidate more links and migration guides here.
1. Migrating from Protractor to Cypress
https://nx.dev/latest/angular/modern-angular/protractor-to-cypress
1. Cypress official doc for Protractor users (placeholder link for now, will be live in the future)
http://on.cypress.io/protractor-to-cypress
[1]: https://www.npmjs.com/package/selenium-webdriver
[2]: https://github.com/SeleniumHQ/selenium/wiki/WebDriverJs#control-flows
I remember giving TestProject a try and I wasn’t fascinated with it, seems similar to a lot of other low-code or no-code tools out there, to be honest I don’t like using record and playback tools, I prefer coding
Record and playback tools are never my first pick, and I don’t see many automation testing companies using them.
They are not flexible as selenium and have never been easy to manage.
I find many flaws in these tools. The maintenance cost with these tools are very high.
There are always many changes in the app, which fails the the tool to playback what was recorded earlier.
Also integration is something these tools have failed in to catch up with selenium.
Selenium on the other hand has now come so far that it has enormous integrations now.
Remote execution, data and keyword driving, parallelization and Behavior driven are also the features which makes selenium versatile.
The only useful case I find with the record and playback tools is when you have a stable app with minimal scope of frequent UI enhancements.
1 Like