What framework are you using for mobile automation?

Someone recently asked on Slack what are the most popular mobile automation frameworks at the moment. I thought it would be interesting to poll people here on The Club to see what frameworks you’re using.

What mobile automation framework are you using?
  • Appium
  • XCUITest
  • Cypress
  • Espresso
  • UI Automator
  • WebdriverIO
  • Perfecto
  • Robo test
  • Other (please explain below)

0 voters

3 Likes

I wanted to vote for 2 options (because iOS & Android) but couldn’t.

1 Like

I couldn’t vote for 2 - wanted to vote for Espresso and XCUITest

1 Like

Hmm try now @maaike.brinkhof & @nicolalindgren, you should be able to choose more than one now that I’ve edited it

Interesting results so far :grin:

How does ROI look for these tools? My experience is that mobile tools are behind web testing in terms of maturity.

2 Likes

That could be a great standalone/new questions @danuk :grin: I don’t personally have much experience with anything except WebdriverIO I’m afraid.

I am trying to look at tool options for testing our websites and how we will be able have automation across all platforms, including devices (our biggest percentage of customer traffic). I am interested in peoples experiences of the available tools so we can review the results, but we are not focusing on the App side as it is a different team.

The Dev’s seem to be really keen for us to use a specific tool, but that seem to have limitations around devices and only seem to really work on browsers/desktop.

Any advice would be great :slight_smile:

2 Likes

Hi Sarah, if you’d like to go all in on mobile, I’d recommend trying out Espresso for Android and xcui for iOS. This would allow you to run your tests on a wide range of devices.

Although those are predominantly used for testing the mobile app, you can leverage the framework to test websites. There are a lot of tutorials to do this, but I’m happy to help if you couldn’t find a good one.

This way you can also add some native testing frameworks to your skillset/toolset.

Good luck!

1 Like

Been testing for on and off over my career, the mobile work started when I used was a “robot-based” framework, possibly an early fork reworked to do mobile testing before mobile testing was even a thing. Lately I am using a totally rewritten pytest in-house framework again, which pulls in Appium and pulls in selenium/webdriverIO, trying to merge them a bit.

1 Like

Interesting to see Appium out on top so far!

Appium is probably only that far up, because it’s used under the covers in many subscription based tools, which is a misnomer perhaps. I just don’t see the brilliant community support that webdriver for example gets.

We use Xcui for iOS and Espresso for Android, having good buy in from developers.
Then running them daily on CircleCI.

4 Likes

Side question on that point, at what stage do you notarize the app in your CI line, and how often does notarize just randomly fail?

Whether I talk about my organization or any other quality assurance service providers, I believe Selenium is one of the most popular automation testing platforms. And when it comes to mobile automation, I think it is only Appium and Selendroid that are two of the most popular test automation frameworks for mobile apps. Especially, when these tools are made to offer all the sophistication of an open-source framework, most of the potential issues are rectified at their earliest with the help of the extremely active community.

Besides, the need, use, and skills of the testers is likely to affect the choice making Calabash and Test Complete two other good options to consider.

1 Like

For mobile application testing, there are so many frameworks and tools which we can use. The most popular ones are mentioned below:

  1. Appium: A popular open-source framework is Appium which is used for automated mobile application testing.
    Developers can use it to automate mobile testing services of native or hybrid iOS and Android applications. This helps to run the test cases using the WebDriver interface.
  2. Calabash: A mobile test automation framework that works with multiple languages such as Flex, Ruby, Java, and .NET.
    For enabling native applications that run on touch screen devices, Testers can use APIs.
  3. XCUITest: XCUITest is a framework that is launched by Apple in 2015 that meant to create and run UI tests on iOS apps using Swift / Objective C.
    Ofently considered the best instrument to create automated tests on native iOS apps. This is known for fast execution, intuitive operation, low flakiness, and easy test management.
  4. EarlGrey: EarlGrey is a testing framework which is Developed by Google and especially useful for creating UI and functional tests. This utilizes Unit Testing Target and for enabling easier creation and maintenance of tests, this is equipped with robust, in-built synchronization.
  5. Selendroid: Also known as selenium for mobile apps for Android. With the help of Selendroid Testers can do native and hybrid mobile application testing.
    Similar to Selenium for cross browser testing, Selendroid can also execute parallel test cases on multiple devices.
  6. Espresso: A mobile automation framework from Google that enables the creation and deployment of UI tests for Android applications.
    Though testing the user interface of an application is essential before deploying it, app developers and app testers majorly use Espresso.
  7. Robotium: A popular open-source tool dedicated to testing android applications only. In this, Jave is used to write test cases.
    This is popular for its ability to write automated black-box test cases. It also automates multiple activities that a tester can execute on their android app
    and takes less time to write test cases.

To do test automation for mobile apps, mobile engineers need to select an effective and profitable test automation framework. This framework is the program that links up to a emulator, simulator, or real mobile device and manages the execution of automated tests.

Appium

Appium is a leading open-source test framework that initiates cross-platform native test automation, as it supports both Android and iOS platforms. It was derived from Selenium in an effort to broaden automated testing functionality to mobile apps.

XCUITest

XCUITest is the official Apple offered automated test framework and is created majorly for testing apps an iOS devices.

Espresso

Espresso is a well-known testing framework that comes built into Android Studio, and is designed majorly for functional testing of Android applications.

1 Like

We use it .

Moderators Edit: Zhou Yixun is the owner/developer of Sonic.

Curios @daniel_lee - you seem to be using more than one testing stack, in different ways. do you have different team members or each or does one person automate against all of these

I am using Appium, but did someone mentioned Playright from Microsoft?:sweat_smile: