I work for a small company, and my role is typically to take our client’s business requirements and translate them into functional. However, I also lead our QA testing effort, and the majority of my experience is in web and API testing. I have very, very little experience testing mobile apps. I have one other tester, but she has limited app testing experience as well.
We’ve been tasked with building a mobile app for our client based on a user rewards program we run for them on the web. We are starting to get to a point in the planning process where I need to think about testing logistics. What resources do you all find really helpful when it comes to mobile app testing? Specifically, I’m trying to work out:
What are the most important physical iOS and Android devices to test on? We have limited resources so I’d like to pick one or two of each and then use an emulator for testing screen rendering, etc. Are there good resources to look at market research i.e. most used devices for each OS?
-What emulator do you use for mobile app testing that you like?
-What resources do you use for general app testing best practices? i.e. testing things like location services, persistent user sessions, push notifications, app badges, and other things that are more used on mobile apps vs the web
As far as devices go, you could pay for Browserstack or SouceLabs, or even have a device farm in the cloud, AWS and Azure have these services.
I don’t have much mobile app testing XP (around six months) but I used BlueStacks as my emulator and Android studio and XCode have good ones, built in - just make sure you got enough RAM and CPU.
The Test Automation University has 3 learning tracks on mobile automation:
Just filter by path and language.
Appium might be a good choice if you previously used Selenium on the web, lot of what you know will carry over.
And in terms of API testing you can use a Proxy tool, like Charles or Fiddler to check the APIs talking to your mobile app.
I’ve not been mobile testing terribly long, or intensively, and use Appium, and can recommend looking around first.
Skimping on only testing on one device per mobile vendor is not really useful, I would push for more resource. What you can do is look at getting 2 devices a tablet and a phone for each vendor. All that will happen is when issues on arise on a device form-factor you don’t have, is that false economy might prevail where people use their private devices to verify customer reported defects, defects you can no longer catch before release. Form factor (phone vs tablet) differences is a common failure area.
The cloud device providers are not cheap, it costs less to buy 2 devices every year, and very soon you have really good cover. You have the benefit of also being able to test lowest supported version and highest soon too. Trouble with owning the phones is you do have to maintain your test F/W. Most folk don’t have the time and end up going cloud.
I think the only reason you want to use a simulator is if you have a very powerful machine, and have an app that is not network/bandwidth sensitive at all. You can rate-limit simulators very easily, but very little replaces actual experience-based testing with a real device on a dodgy wifi or 5G while on a train. I’m about to do some reboot of my mobile framework myself, and am also looking at possibly swapping away from Appium. Depending on if the app is native or browser based, mileage will vary a lot.
Lastly mobile apps are harder to test than you think, and getting some buy in to resource this better, might prevent it sliding later on. Although, from the looks of it, you need another test engineer. Seems to be a recurring trend though Nicole.
What are the most important physical iOS and Android devices to test on? We have limited resources so I’d like to pick one or two of each and then use an emulator for testing screen rendering, etc.
This depends on your target audience. If you already have analytics from your user base then I’d recommend reviewing it. Otherwise base it off the demographics and see if you find the commonly used mobile phones
Are there good resources to look at market research i.e. most used devices for each OS?
I hope my previous answer helps!
-What emulator do you use for mobile app testing that you like?
If you are able to use browser stack’s device farm, or equivalent, I’d recommend it. Good value for money.
-What resources do you use for general app testing best practices? i.e. testing things like location services, persistent user sessions, push notifications, app badges, and other things that are more used on mobile apps vs the web
Don’t think there is a single place for this, but if you’d like I’m happy to answer more specific questions and help you get started with the journey. Ta!
