This question concerns current trends or thoughts on testing Web apps (so specifically Web applications which have optimisation for using on Mobile devices) and device coverage folk tend to go with.
For website testing, I will honestly say I’m a bit of a sceptic when it comes to testing on a broad range of devices.
Do people still test mobile sites on a broad range of devices? Has anyone ever found a bug that only appears on a specific device (and by this, I don’t mean a bug on Iphone specifically Vs Android, I do mean for example a bug that was only seen specifically on a Samsung S22 and not others). Was the device the issue or something more like the screen resolution (i.e. is it Screen res that is the issue, not the device per se)?
I do tend to limit the testing to a few examples to start with - principally an android, and an apple and then maybe go with the regular and the larger “pro” size devices. Then go from there if we find issues.
I’m interested in how deep people tend to go in this area.
As per your question:
“Do people still test mobile sites on a broad range of devices?”
They usually don’t, and it shows.
I am mostly focused on mobile apps, but there are a lot of similarities. I’ve found a lot of bugs on websites for a specific combination, quite the same as with mobile apps.
It’s not the Samsung Galaxy S22 that’s a problem. It’s a combination of:
1080 x 2340
Android version
Samsung Internet version
or
Android webview version
In the case of iPhones, then yeah. A specific device with a specific iOS version is a problem most of the time. iPhone XR and iPhone 7 are still a mess to deal with; there are a lot of problems with those. Don’t forget about iPhone 6s and 6s Plus.
And don’t get me started with foldables, Android Go and exotic Android brands.
In short, browsers, like any other app, use OS API, which can change with each version.
Not to mention support for codecs, push notifications, graphic format support (WebP, WebM), or security features. Browser engine updates can also come with OS updates.
So, just like the exact version of the app (1.0.1) works differently on various OS versions and different devices, the same can happen with browsers, which can impact multiple aspects of handling a website despite using the same version of the browser.
I’m actually quite fortunate (ish) at the moment as the apps we develop are more b2b and we can have a say in the browsers we “officially” support.
For me it was a nightmare too - testing web browsers on the mobile phones. Each Android version (from 12 for all) was somehow different from how we started browser to other things.
iOS was horrible - we had iframes and other iOS typical super secure things which did it almost impossible to automate with Appium. At least stable
