Does anyone have any experience of testing a system that sends a verification email to a user?
Iām trying to think of the best way to go about this. I have an automated journey that creates a new record for a user and we are currently looking to add an email verification step into this at commencement of the journey.
Iād like to imagine there was a way of mocking this part of the journey, ideally iād like to avoid having to hardcode a permanently valid email address.
Is your question about testing the confirmation system, or about automating a task?
If itās the latter then it depends on what tools/tech/language youāre using, what level youāre working at, and how this check supports your testing (what exactly youāre looking to check). You could code a UI check end-to-end that creates a known user record with a known email and then have a service that (based on what triggers your verification emails) pings emails back to your test system, where the contents are read. Youāll need some sync code for this, obviously, youāre testing a distributed system. This checks that it was received (in this particular setup) and you can check anything you like about the email contents - if thatās what you want to know. It doesnāt check a whole lot of other things, but thatās a matter of choice on your end. Maybe you donāt even want to check that many things.
If itās the former then youāre asking a wider question. Why are you verifying the email, and how important is it? Some stuff might include:
Security. Can I get around the verification step? What can I acheive without verification? What if I use a disposable email address, do I need to take steps to block them? Can my system be used to spam someone by requesting verification emails over and over to an email I donāt own? Can I DoS the verification system? Can I SQL inject the system via the user record? How long is the validation link valid for? Does a new link invalidate the old one? Can link URLs be reused - how are they tied to the user and what does that reveal about the system? Can I use a bot to create accounts? Can I verify other peopleās accounts by editing the return URL? What does the return URL expose about our API, and how can that be used?
Scalability. Does it work under stress? What about under load - what happens, for example, if I break the message queue that sends the emails? What if the email server goes down - whatās the user experience?
Functionality. Does it send? Can I receive the verification? What if the link is clicked twice, or 20 times in frustration? What if someone spams the āresend emailā button? What if I refresh the page that caused the link to be sent - will it quietly send another and invalidate the last one?
Usability. What if the email goes into spam? What if itās not received quickly? Do I have to type in verification numbers manually, which can be a pain, especially for people with related disabilities? If some of my customers are not terribly tech savvy then have I given them suitable instructions to verify this email? As a user is it worth the effort of verifying my email?
Secondary Purposes. What can go wrong when we assume that the email really is valid when itās not? When should we use it as the only means of communicating something, such as a policy change? What is the email address actually used for, and what if it starts bouncing mail?
Up/downstream dependencies. What if I use a really long, badly formatted or otherwise invalid email/name/body string? Are the email sender, validation system, database and UIās interfaces all data compatible (simple example: email too long in UI to store in DB). Where is the data stored and used, and what can I get past the filters?
And so on.
PS. There are third-party email verification tools abound online, I donāt have experience with them, though.
Thank you for the in-depth response. Its about both I suppose, the idea behind the question was about strategies for integrating with automation, I wasnāt very specific.
A confirmed email will be required for the user to continue through a process, this is currently not included in the automated regression. Iām not concerned about the āwhatā to check for this bit of work for now, only the method of moving through the journey.
With the intent being essentially to simulate a user responding to an email confirmation email, the question becomes how much would I want to wrap in automated tasks and how easy would that be. It would be straight forward enough for the script to start on the users confirmation landing page passing in fuzzed auth details and continuing from there but Iāve then bypassed the functionality part of the email client that you rightly mentioned above.
Iām coming to realise however that this question may be beyond the scope of such a forum due to it, as you say, depending upon various things. Namely the chosen email client and whether there are any hooks/libraries that would let me interact with it in a test in the chosen language (currently js but may change to Java).
Primarily, I was simply curious if anyone has had to implement such a system test before.
Iāve always done the email verification part manually, and used workarounds like running SQL updates to get the result I needed where I had to automate user registration.
That said, Iāve used Guerrilla Mail as my āfake-realā email tool of choice for a while now. Itās an HTML interface where you can use a chosen email or the scrambled version GM automatically generates. The emails themselves are deleted after an hour, so itās relatively secure, and any address with the right domain (they default to @sharklasers.com) is automatically valid.
If you absolutely had to you could probably use that to pull the information, but honestly, the more third parties you run through UI automation, the more fragile your tests are going to be.
I immediately think ālikelihood of failureā is low? In which case, and you allude to this, mock it. Depends on your stack but should be as simple as Kate suggests, stub with a script (SQL update) to service your automated test suite.
If there is risk of failure, perhaps it stacks up as a high priority to fix and subject to manual checking until that is so.
I usually find any system which sends a verification email has a configuration setting for the email server. All the systems which I have been testing can be configured with SMTP as their email protocol. In these cases I just find a simple SMTP server, configure the system being tested to use the simple SMTP server I set up.
For example, you could use https://github.com/Nilhcem/FakeSMTP. This is a Java project. It creates a jar file that you can run on any system. You then configure the system under test to use FakeSMTP as your email server.
I usually have the output directory for the FakeSMTP be something my test automation can access (an UNC share if it is Windows or a SAMBA share for Linux/macOS). Then my test automation can access the file share and read the emails from there (no need for an SMTP client to retrieve the email).
Another one, with a web interface, might be https://github.com/djfarrelly/MailDev. I found this by searching for āSMTP server with web interfaceā.
It has been a while since I did this kind of testing. So I donāt remember the exact SMTP server I was using.
Iāve also set up SMTP servers on UNIX systems but configured them to queue up the email in a spool then never send it. The emails would then sit in a directory on the server. I could ssh to the server and read the email. A little more complicated to set up if you havenāt done this sort of thing before (used to be a UNIX system administrator; so setting up an email server was second nature).
I will add this on - as I donāt see it mentioned yet.
Manually testing email - Iāve had good success with MailTrap https://mailtrap.io/
Basically, you configure your app to use it as the SMTP server in your Local/Dev environment, and it captures ALL mail being sent, to any email address. OR you can configure it it lots of ways to route email based on address/domain.
Iām not sure how well this integrates with automation, for but manual / exploratory testing itās great!
Mailtrap is really good for automation as they have a nice, easy to use API. Iād definitely recommend this as an option if you have control over the SMTP settings for your project.
Other options Iāve seen are usually paid for such as Mailinator but the pricing was a bit steep.
I also had a go at using Gmail via IMAP though you can get held up by the delay of the mailbox refreshing and I think Gmail have some T&Cs which may rule this out if youāre doing anything at scale.
Iāve used https://www.mailinator.com/
You could send an email to any random string@mailinator.com, and then find the email on mailinator by searching for that random string.
The enterprise plan makes it secure with your own domain (others cant see the emails), and also gives you access to the api, and access to libraries in various languages that interact with this api.
Email feature is one of the most important feature of every application. It provides us benefits to get members notified for updates, approvals, tasks and etc. Its importance is very high in e-commerce domain.
As this is very important feature, hence its testing also become very important for every software testing services provider. We need to consider following factors while testing email confirmation:
Ensure that content is accurate and reflecting correct information.
Content of Email verification code should be properly visible.
Confirmation Link/button should be properly working.
It should take the user to correct portal.
Email should be successfully verified when it reached the application page.
It should not take much time to reach application.
Appropriate fonts and texts should be used in the email.
Application links should be correctly displayed and working.