I’m testing push notifications on our native mobile apps and currently I’m brainstorming test cases.
We are implementing this from scratch so I wanted to know how you guys do it?
Here’s my list of ideas, feel free to suggest yours:
Try to receive notification with logout state
When app is:
in foreground
in background
not launched
force stopped
when device is on:
Wifi
Data
with user who is logged in to 2 devices
when device is:
on the aspect of device on wifi/data, I’d also test how the notification behaves if device is offline (from internet) at time of intended notification event, then after some time it comes back online (whether on wifi or data)
The intent is to see if the notification still comes, though delayed because device was offline, and also to assess the delay. Does it come right after the device is online or is there additional delay? This additional delay might be based on how your app/service implements the notification logic on the cloud/server side.
Additional variations, though not really necessary, would be to test how the notification is handled if device was powered off then on later, at the time of expected notification. Or device rebooting at that time, etc.
What happens when the device is low on battery and a push notification is expected?
This would also be regarding whether device OS has low battery mode turned on or not. Depending on how low of low battery you are testing. Usually in those modes, notifications are disabled on apps, and the apps may not be running actively.
And if turned on, what happens when you use the exception feature on some device OS to exempt the specified app from being disabled (for use and notifications), do you then get notification after the temporary exemption?
There were a lot of good and inspiring ideas
I don’t know if this is part of your question, but when I test notifications I also often come across deep links that take you to a specific part of the app. This introduces a lot of additional risks and test cases.
And I am not talking about the possibility that you can choose from some options in the notification and the app has to remember that and react (behave) accordingly.
These are also big areas of testing for me.
Part of me though is wondering if all of those things are within your apps control.
When you say implementing from scratch does this mean you are building your own notification service within your app or are you using a 3rd party service?
The former will expand your testing as you may not have used standard protocols for example, but the latter its maybe more a case of testing you have implemented and configured the service correctly. With 3rd party implementation there should be certain industry standards that apply and it can get a bit grey if you are doing their testing for them as opposed to the risks your own implementation could introduce.
Devices themselves can also control a lot of things, multiple variations of do not disturb for example, so is the risk that your app may be able to bypass device settings in some way or is the testing to confirm it conforms to expected device setting behavior. For example if your notifications were somehow flagged as critical emergency notifications that perhaps could bypass general phone configuration.
If you are building your own its worth trying to find a standard to work against.
The other part is how your app reacts to attempted notifications, consider the bad internet connection and user does not get the notification for some reason, how does your app deal with failures to receive, is it aware its not been received, will retry later?
I recently was setting up a Android for DUO authentication, device and the push notification pops a screen on the app if you have it open/foregrounded with a “deny” . “accept” choice, but when the app is backgrounded or not foregrounded/open just tapping in the notification tray accepted the connection, which was bizarre, so yeah, anything can happen in the push notifications land if your app is non-trivial and has multiple intent handlers. Apple is completely different for UI once again.
Well frankly speaking, yes.
But since this is the first time our application is getting push notification, I thought it would be good to rest thoroughly enough.
I’ve been trying to build out a lot of these for a new Moropo video I’m doing on push notifications - but annoyingly, I don’t have a decent app to test against.
