Test Ideas for a Login Screen

(Heather) #1

About 8 years ago Ministry of Testing (then Software Testing Club) crowdsourced ideas for testing a login page. The responses from the group were documented, I’ve attached them to this post. You can see the login screen on the second page.

I think many take login screens for granted as having an obvious set of tests. There are many different things to consider with login screens. They are usually the first thing users will meet for your application so giving a good impression at this point is pretty important!

Would you add anything to the listed ideas? Maybe you’d elaborate on or extend some of them a bit more.

crowdsource-testideas-loginscreen.pdf (551.2 KB)

Proposing The Case For Alternative Authentication Methods
(Miranda) #2

The information covers most of the testing around the login screen. I would expand on data and security/error handling.

The data field may allow characters that are not handled correctly by the database without proper encoding. Copy and pasting of the password is mentioned but it’s not clear that testing could cover copying from the field and pasting in notepad could show the password in clear case versus copying the password from a MS Word document, that include unintended hidden characters.

In regards to security and error handling, some applications lock the user out of their account after X number of tries. Emails may be sent to the email on file notifying user the account is locked. Procedures for unlocking the account should be tested (i.e., lock clears after 30 minutes, after user resets their password from a link, etc.).

(Rosie) #3

I saw this post and thought it might give inspiration on ideas on how to test login screens - http://www.webdesigndev.com/mobile-login-forms/

(Heather) #4

I saw this in (I think) the MoT blog feed recently too http://startesting.it/login-screen-testing/

(Michelangelo van Dam) #5

Testing a login form seems a trivial process, but to keep up with security requirements and usability efforts, this trivial process has got very complex.

A few things we test for that were not mentioned in the PDF guide, are focused on testing the security of the login form.

  • data overload: how much data can we submit through the form so the system fails. We copy a complete encyclopedia (~4GB of data) and paste it in the password input field as there is less validation on input.
  • common password lookup: after the multiple breaches that occurred in the past decade, millions of passwords have been analyzed and a list is being published each year with the most common passwords (see WikiPedia). Is the login form performing a lookup in this list?
  • JavaScript off test: since most forms are now making asynchronous posts using JavaScript for input validation and password strength checking, is the form still working when JavaScript is disabled?

When two-factor authentication is being provide, the tests are extending even further.

  • test with invalid token
  • test with valid token
  • test with invalid backup code
  • test with valid backup code
  • test lockout procedure
  • test recovery process

And when teams are involved, roles of team rights for account restore procedure needs to be tested as well.

I hope you like this addition and find it useful.

(Michelangelo van Dam) #6

This whole discussion got me thinking about the next step in authentication: two-factor authentication.

This form of authentication adds another layer of complexity into the mix as the secondary factor value can only be provided through another system.

I wrote a blog post on this subject and I love to receive your feedback on it.

(Andrew) #7

Jared Spool (aka God of UX) did a brilliant talk on what he called SUX: Security UX. I’d highly recommend watching the video or reading through the slides. To quote "If it’s not usable it’s not secure."

(Geoff) #8

This is fab, I have used a Login Screen print out as an exercise to give candidates for test jobs for the last couple of years. Not as a pass/fail but as a quick measure of someone’s knowledge and ability. My “key areas” list is pretty much the same as the original document, but also has responsiveness and performance on top.

(Michelangelo van Dam) #9

I was reading Wired over the weekend and came across this article of Mister John Null. Just like most Irish people (e.g. O’Donnel) he too is not accepted as a person when signing up for online services.

The article: https://www.wired.com/2015/11/null/amp/

Question to all you testers out there: is there a set of names that you test against to ensure they are accepted by the system you’re testing for?

(Heather) #10

I have a few sets I try but I’m definitely guilty of not enough coverage of names. My usual set includes accents (fada in Irish language) á é í ú ó names with Hyphens e.g. Anne-Marie Reid-Duff. I’m amazed with the amount of forms that accept hyphens in one name field and not the other. Apostrophes are another interesting one D’arcy as a first name for example.

I’ve also seen cases where you could sign up with all of these things but you couldn’t sign in with them. @dominic1 has a good set of examples that he uses too if I recall correctly :smile:

(Michelangelo van Dam) #11

Not sure if it’s of value here, but I have here a small list of names I use for testing with the expected result from it:

  • Michelangelo van Dam: testing last name “van Dam” is not capitalised as “Van Dam”
  • Dr. John M. Smith, Jr.: testing the dots are accepted for abbreviation of prefix, middle name and suffix (one name field testing)
  • Kevin O’Donnel: testing that an Irish name is accepted and not turned into “ODonnel”
  • Jane Cock: testing that the name is not flagged as “improper”
  • Sébastian Gruße: testing that some foreign characters are not rejected or removed making “Grue”
  • Kees-jan Klaassen: testing hyphens are not removed turning first names into “Keesjan”
  • محمد أكبر: testing right-to-left direction and unicode characters

After reading the Wired article, I added: John Null: testing Null is not considered a reserved word.

(Jesper) #12

there are a good collection of tricky names in: https://gojko.github.io/bugmagnet/

even more tricky strings here: https://github.com/minimaxir/big-list-of-naughty-strings

(Michelangelo van Dam) #13

Hey @jesper, thank you for these links! I’ve just bookmarked them, thanks a bunch!

(Dirk) #14

The ‘big list of naughty strings’ I knew, but not the plugin. Thanks! Seems handy also to just quickly input valid data (I use Autofill to quickly fill in long forms that I encounter often, Bugmagnet seems usefull to just quickly generate a name or Lorem Ipsum text).

(Stefan) #15

I’m thinking of the context first. What are we talking about here?
What I know so far is that there’s a login screen:
Login = an act of logging in to a computer, database, or system.
Screen = a flat panel or area on an electronic device such as a television, computer, or smartphone, on which images and data are displayed.

There are different types of ‘‘login’’/authentication: logon, login, sign-in. Are they under the same discussion?
Do we just consider a fully virtual environment or a combination with real device, physical actions also? like eye or hand scan, badge or card, keyboard code buttons; interaction between multiple devices or platforms? have a token to login to bank account, have a key/badge and a code to start/stop the house alarm, IoT + other platform/device…
What about the device type used? pc, mobile, tablet, ioT, token, medical device, other kinds of hardware…
OS dependent or or is it built/linked to the device?
Login screen type?: pin, command line, text-field, dots, fingerprint-reader, signature…
Application type?: hosted, server, web app, mobile app.
Online or offline login?
The list of questions can get pretty big…and in my opinion these can be considered tests also.

(Heather) #16

A blog post I noticed in the MoT blogfeed today More tests for login forms

(Paul) #18

Regarding testing of field validations, we know that exploratory testing isn’t amenable to automation however it would easily be possible to create an automation framework containing the types of test data used in Bug Magnet, link it into (say) Selenium and fill various form fields with them to iteratively test for errors or some unexpected behaviour. When an error is found a further check could be made on test data of a specific category (i.e. names with accents, extreme or out of bounds numbers).

It wouldn’t take much effort to create it and I need a hobby anyway…

(Heather) #19

That would be quite the hobby :stuck_out_tongue_winking_eye:

(Paul) #20

Haha @heather_reid it may not be too bad actually. I’ve done a few vaguely similar things quickly in the past -

The Aussie Data Feed for a fake test data creation library in Python called Mimesis (formerly Elizabeth)

A tool for creating fake customer data in Excel and an Excel spreadsheet with various fuzzed data fields already in it

(Heather) #21

I retract my previous statement :grin: that’s quite impressive.