Simple Android automation tool?

Hi, I’m new around here so apologies if I’m asking a question that’s been answered before.

I’m a seasoned tester but new to testing mobile apps. I have an Android app that I want to set up a simple automated flow with but I’m unsure of the best way to approach it. What I want to do is:

  • Push a CTA buton
  • Wait a couple of seconds
  • Check for some text on screen
  • Notify me somehow if text is found
  • Repeat every maybe 15s

The app is running on an Android Studio emulator.

A quick Google turns up Appium as a popular mobile automation framework but for the above it seems like overkill. Even just the set up looks like a total PITA.

Does anyone have any suggestions for something a bit simpler to do this test?

4 Likes

/Edited response. follows
Eeeeew I’m just quite terrible at reading back what I write - So I’m re-writting what was a terrible answer to Calum’s question.

Anyway, sorry about that @calumjcraig , welcome to the Ministry of Testing. Where often technical questions of this nature with complex answers are not asked or dealt with that simply. It depends on a number of things how to help you. What follows is my personal opinion only.

  1. Ask for lots more time.
  2. Ask yourself why do you want to automate one interaction? What is wrong with a manual test?

Unless you plan to do much more test automation, don’t automate this. Just don’t. There are many approaches and very few of them scale well, and very few of them are simple. Also testing mobile apps on a simulator is a good idea, but it’s no good as a test for what you are trying to do here at all. I kind of jumped the gun initially. But I have 5 or 6 questions.

  1. Is your Android app a native app, or a browser skinning app?
  2. Is the CTA an Android notification or a screen in the app?
  3. Is the CTA triggered from a push notification, I’m trying to understand how the CTA is different from any other normal UI interaction you see.
  4. Why is a simple screen-image compare deemed good enough for you? What is the end-to-end steps for this test-case, surely the CTA is to generate revenue, is there another way to check that?
  5. Why do you want to use a simulator not a real phone, it’s normally not relevant, but I’m trying to understand why you want to not use a real phone.
  6. Why exactly is iOS not in scope? Is it merely a package signing/provisioning problem that is holding that back or a business need?

Yes mobile automation is a PITA. It’s too much work for only one test-case. If you still want simple, go with a paid cloud solution, they are rather expensive (especially if you only have 1 test), but all you need to do is get your IT admin to help set up a VPN tunnel with their cloud and they will help you get up and running. The cloud solutions use appium under the hood, but they remove the pain. If you still want to automate, do tell us a bit more about your business constraints and goals.

Disclaimer: I don’t use a cloud mobile solution for other security reasons.

4 Likes

If you are already using Android Studio, why not try using Espresso ? You can also plug a real android phone into your laptop and use chrome (chrome://inspect#devices) to spy on your app. If there are plans to expand support to iOS, then you should consider using Appium (with language of your choice).

2 Likes

Thank you, that was the sort of suggestion I was hoping for. Will look into that .

For my sins in giving a nasty answer that got us both grumpy, a more considered set of options. Espresso is a good option if all you want is Android automation, but without knowing what kind of app you are testing, hard to estimate required effort. It’s not always relevant, but trying to guess what your test cases really does helps get a handle on the effort you want to spend.

A big reason Appium is tricky, is that it’s super hard to install and super hard to get working with the Apple automation drivers. If you definitely never want to support iOS, Appium is still easy to use for Android, but it’s plagued at the moment by being in flux between version 1.22 and version 2 not being officially out yet. So go with @greetomosquito suggestion to go Espresso. Appium is slow, and although people say it’s flakey, that’s often just poor test coding, which you won’t experience much of since you know about race conditions already. Appium tests are really like writing web browser tests, it uses webdriverio as you know, and it polls. So it requires quite a lot of scaffolding. Scaffolding that’s not worth it if all you have is one test case, and which you get for free in Espresso. If you later do want to support iOS, you will want talk to the developers and look at the app stack and see how much code sharing between the OS/s exists. Some teams write tests in xcuitest for apple and duplicate the test in espresso for Android and are happy to keep doing that. That’s a valid option, but I opted for writing all of my tests once only, and dealing with workflow variations per OS, and went with the pain of Appium. I regret it often.

Android platform is deceptively easy to automate until you get to OS version 10 where they started to close down the security holes that made app testing on Android easy. Knowing if it’s a browser app or a native app just helps me get a better idea of if your app has any OS integrations. Things that, if your app is non-trivial might change behavior in Android 11 and also in 13. I found that Appium did not help me much in dealing with Android 13, not with vendor variations. If your app uses notifications etc, knowing who your favorite phone maker is also helps later on.

2 Likes

Thanks for that - lots of good food for thought. Yeah, what you say about Appium kinda confirms my initial thoughts that the effort involved wouldn’t be worth it.

For reference it is a native app, running on Android 12. I do the vast majority of my testing manually using the Android Studio emulator but just want to automate this particular test as I want to hit the button and check for text several times a minute for hours on end.

I could use a physical device but the only phone I have is my own personal one so would prefer not to unless there are clear advantages over the emulator.

Sounds like I need to look into Espresso in the first instance.

2 Likes
  1. Is your Android app a native app, or a browser skinning app?
  • It’s a native app, not sitting on top of a browser.
  1. Is the CTA an Android notification or a screen in the app?
  • It’s a button in the app.
  1. Is the CTA triggered from a push notification, I’m trying to understand how the CTA is different from any other normal UI interaction you see.
  • No, it’s just a standard button press - maybe I clouded the waters using the term CTA, that’s probably not relevant.
  1. Why is a simple screen-image compare deemed good enough for you? What is the end-to-end steps for this test-case, surely the CTA is to generate revenue, is there another way to check that?
  • I want to test for a particular outcome - there will be a unique word displayed on screen which I figured would be the easiest way to test it.
  • This is the very first stage. I may want to build up a more developed E2E test case in the future but not at this point.
  1. Why do you want to use a simulator not a real phone, it’s normally not relevant, but I’m trying to understand why you want to not use a real phone.
  • As I mention below I am not against using a physical device - if there are advantages I may consider doing so.
  1. Why exactly is iOS not in scope? Is it merely a package signing/provisioning problem that is holding that back or a business need?
  • iOS is out of scope because this is a pet project for me on the side, not my day job, and I only own an Android and therefore only have access the apk in question.
1 Like

Hiho @calumjcraig! :slight_smile:

If your requirements are that simple, then check out Maestro: GitHub - mobile-dev-inc/maestro: Painless Mobile UI Automation.

Quick and easy, it should be great for this situation.

I use it mainly for PoC for my clients and fast-check whether something is easily automatable, and it works surprisingly well so far.

Just remember to run it locally (or on CI that you control); using their cloud is not free.

4 Likes

WOW, oh my. I can see why you want to automate this check in a hurry and with low effort outlay now. Espresso would be my route, for something like this since I’m embedded in my dev team, so not too much learning solo if a developer of the app can help a bit.

I had a look at the Maestro app last year that @pwicherski mentions, because the boss wanted us to look at cloud, and it sounds like a good fit for your use case.

A few things on your side, is the native app. The browser-based apps are easier initially but those get tricky later on once you ask security/Multi-Factor Authentication and cache questions. Ah, so your intent is to build out as you go, but I would defo use this as a chance to create a single-use tool that covers just the Android use-case. A simple test tool that is easy to set up and maintain will let you learn about all the pain that is mobile. Because Espresso tests will run without installing the full Android Studio, it will be possible to more easily add your test into a CI/CD if it’s headless.

A few of the dragons are

  • iOS automation requires that you get up to speed with testflight
  • Android has much easier to use commandline tooling than iOS
  • you need to add your test apple phones into a “provisioning profile” if you want to manually install packages ever
  • Android builds and binary signing are tightening up and catching up to Apple. Basically means you will want to get help with gradle files and getting your build system to actually build custom targets in your future

A few mobile bootcamp resources that might not be covered in any other threads. If you have MOT Pro subscription, there is an actual entire recorded mobile testbash. You can binge-watch the entire thing before you dig about for previous recordings from other test-bashes. Which is brilliant at getting into mobile, the last event actually had a speaker cover each of the major tools in great detail. It’s the big reason I have a MOT subscription (also I hate actually going to conferences).

I’ll try dig up more quality links, but this feels like a longer term “eat a frog” (aka Book Summary: Eat That Frog by Brian Tracy | Sam Thomas Davies) which you really want to put off for a bit, and do a small throwaway spike initially.

3 Likes

Don’t you need also a Mac for iOS automation (development and execution?) while you can do Android on nearly any hardware?

1 Like

Sorry for my tardiness in replying - other things got in the way for a couple of weeks and I’m only just coming back to this. Which, as I said is very much a pet project so my day job takes priority.

Thanks for the pointers - I will come back and let you know how I get on.

3 Likes

OK, haven’t had a lot of time to write tests but I’ve installed Maestro, plugged my phone into the USB of my laptop and… it just works.

I think this will do exactly what I want it to do and the tests are super simple to create. Great tip, thanks again for the responses to my query.

Nice one, @calumjcraig. Good to hear you’ve found a solution to experiment with.

As a side note, I’ve added Maestro to the Ministry of Testing Tools Directory.

1 Like

One of my Devs also wants to look at Maestro, will encourage them to give it a go now.

2 Likes

I was amazed how easy and quick it was to get up and running.

The official documentation isn’t particularly detailed and I don’t think Maestro has a huge range of functions but as the title of my post said, I was looking for a simple automation tool to do a specific job and this definitely seems to fit that.

1 Like

:thinking: Perhaps this is a possibility to work with them to help improve their documentation. So the next person has a better experience.

1 Like

After a look, Maestro suffers from wanting you to write plugins to do most things. Also suffers from inability to decompose workflows and make scaling up to running multiple branches or paths in your app without training it in each path first. Maestro wants you to write separate tests for tablets and phone forms. Yes, tablet and phone differ more than you think on iOS and Android. We don’t want to have to write 2 tests for example when a tablet is in landscape and again when it is in portrait unless the test case demands a requirement for that. Layouts must just work in all tests.

Did not fit our SaaS business testing environment need where we really need to be able to build complex scenarios and get device and form-factor coverage and writing test cases once for all targets. But we still learned a few rough guidance pointers in the exercise and are looking at a new facebook tool called idb (adb for ios).

1 Like

So what was the final result @calumjcraig ?

Appium or Maestro?

or maybe you found something that will suite your needs way better than those two?