Hi guys. I am testing the scenario where users are not getting 2FA when they are trying to do any banking transaction. they said waited for a long time no success. I tried to cover all scenarios but I was not getting any issues with that 2FA SMS. how can i test this feature where I can encounter the issue.
1 Like
I don’t think this is a testing issue. To me it sounds like a monitoring issue. You need to have enough logging in place to be able to verify that your API calls to the SMS gateway (which I suppose is 3rd party) are sent and received OK, or if not, what are the reason codes and error messages. Then you’d be able to determine whether the issue with SMS messages not being delivered lies on your side or on the provider’s side.
1 Like
Just to give you one possible scenario that I personally encountered: we were receiving customer’s phone numbers from a partner’s database and they made a change in their API (without notifying us) which caused them to stop sending the country prefix with the number. We passed that number to the SMS gateway and then obviously the SMS sending failed because of the missing prefix.
We would have never noticed the issue and found the root cause if we didn’t have proper logging and monitoring in place.
1 Like
A solution to handle this problem is to test in production on a daily basis.
For API testing, approval tests could e considered. E.g. this phone number has the right format, so it should be the next day.
Another way to handle changes is to make good appointments with partners. If things will be changed, then your company should be informed ahead of time. This way the impact of a change can be reduced for your customer
1 Like
Well, I’ve encountered that issue in the Real World. In my case, it was because I had a poor mobile signal where I live (1 bar reception at 3am was the best that mobile service provider could deliver) and SMS delivery is actually at the back of the queue when phone system connectivity is restricted. (I solved this by switching to a different provider who delivers better service locally.)
Quite how you replicate this in test is another matter, beyond finding a location available to you with similar poor service, or putting your mobile device in a shielded physical container or room.
2 Likes
SMS is not a guaranteed delivery system. Depending on the country or network, results may vary. I still recall 20 years ago using it for a demo in process control and finding that in some networks an SMS is actually damn fast, but in other cases, the mechanics of the network means that SMS messages get stored in a database wherever the subscriber was last online. When a subscriber connects to a tower, that tower might not have their “lost” messages, and at that point the network updates, and after a time you might get all your messages about 3 hours late.
Which is why a UI with an option to send another message after a small delay of 2-3 minutes is pretty much standard. Users will soon learn to go and switch their phones on, place the phone near a window and then press that button to try again. As for testing, you cannot test that scenario, it’s not part of a thing you control or own. BUT, like @han_toan_lim says, have a test that runs either on a cron job or in your CI system to verify as much of the plumbing as possible. The cost of an SMS becomes negligible when you count the business cost, just remember to delete all your SMS on the target test device, that can be embarrassing if you don’t.
2 Likes
That’s exactly the situation with the phone network in the UK with respect to unconnected devices. In my case, I live down a private driveway some 100 metres from the public road, where there is adequate signal. I am on the edge of a mobile cell, and my old provider’s mast was in one direction where I am in the shadow of a hill. I could get signal if I walked the 100 metres to the road, which was usable in an emergency but little else (and when my phone’s battery collapsed and couldn’t be recharged, that was the point when I went out and bought a new phone on a different network). My new provider’s mast is in the opposite direction where the terrain is more favourable and signal is stronger and constantly available.
Which suggests to me that one test that ought to be investigated is what happens when the user receives a 2FA code that has either expired or has been overtaken by a newer one that they have received . Does the system reject expired 2FA codes if they are presented? What does the user see if they receive and try to use an expired code? Or if the user has requested a resend of a 2FA code, are all the old codes invalidated? We know that they ought to be - this is basic security stuff, after all - but as testers we should not be taking these things on trust or what system designers say should be happening.
2 Likes