Invalid password vs empty password in login form

Hi :slight_smile:

To begin with - I am a newbie and the more I think about it, the more I am confused.

In terms of writing test cases - should we consider both below test cases if we verify login functionality ? ( We don’t consider error validation messages)

  1. Verify that user is unable to login with valid username and invalid password

  2. Verify that user is unable to login with valid username and blank password

Isn’t it just a repetition and unnecessary additional test case when considering blank password, if we already consider the invalid one?

Isn’t blank password and invalid password processed in the same way?

I mean: is the scenario that we are unable to login with valid username and invalid password, but able to login with valid username and blank password even possibile at the same time?



Firstly, welcome.

As for your question, depends on the feedback to the user, but you may have already answered your own question.

If you attempt a login with a blank password, in an ideal world I would like messaging such as; ‘Password cannot be blank’

If you attempt a login with an invalid password, in an ideal world I would like messaging such as; ‘Invalid password, please try again or reset your password here ’

If both of the above is true, and you see value in checking the correct messaging shows - then yes, two separate checks.

If there is no feedback to the user in either case, I’d challenge that - even for a back-end system that is not a good user experience.

The messaging in both cases might just be ‘invalid password’ - Which is fine I guess, and for you means one test case, but you’re checking both at the same time as both invalid and blank passwords are simply deemed invalid.

Personally, I would be looking at automating this sort of check as having one or two assertions like this is trivial, and will take the pain away from you doing it manually each time you get a new version of whatever it is you’re testing here.


Definitely not! This would allow hackers to enumerate existing user accounts.
You want a generic error message stating ‘Username& Password combination is wrong’
You would never want to say the username is wrong nor the password, this brings in a lot of security risks.

Just wanted to highlight this :stuck_out_tongue: I hope it doesn’t sound to harsh? XD


Not harsh at all, a more generalised ‘username / password is invalid’ would be better messaging, I agree!

1 Like

A blank password might a very specific case of an invalid one. I will consider I an extra case.
And there might be others cases too. E.g. encoding like ASCII vs UTF.

I also heard once about a bug very a user was able to logging on his account with someone else’s password. The application just checked that it was A correct password (one out of all available), not THE correct password to the account.

Or is maybe the user logged in to someone else’s account by using that accounts password? Could also happen.

Isn’t blank password and invalid password processed in the same way?

Finally I suggest: Ask your devs for the implementation details (or check it by yourself at the code if you can). Someone outside of your project can’t answer that question.
Also discuss with all how much risk is at this topics and what effort you should spent reasonable.
Be brave to stay to your assessment as long as you are convinced. Don’t find up to easily, but give up if others have striking arguments.

1 Like

First I’ll just say if you stop looking at testing as a series of lines of checks and more as an exploration based on your understanding of the product guided by risk you can achieve more with much less writing, reduce the cost of upkeep, avoid going over old ground again and again, plus it’s more engaging and fun and you might find some really good problems that the test cases don’t account for. You’ll find that you can do more interesting test cases if you don’t write them down in that way. Move towards a checklist of suggestions rather than a case-by-case basis. This deformalisation will let you do more testing and less writing with more freedom to ask interesting questions of the product. What you actually consider when you test is, let’s face it, not determined by what’s on the test case list but with your real-life interactions with the product.

You pose a lot of interesting hypotheses. If all of our hypotheses about the product were true then there’d never be any bugs in it - after all the product is developed to work and one big hypothesis is that everything’s okay because we made it to be okay. This is clearly not true, so we must learn to distrust our hypotheses and test them to turn them into more robust theories of the product.

To the specifics:

Isn’t it just a repetition and unnecessary additional test case when considering blank password, if we already consider the invalid one?

Maybe. Perhaps some part of the code that expected to receive an input simply falls over when there’s no input provided. Perhaps it is supposed to display a different message…

( We don’t consider error validation messages)

Don’t you? Because if it was full of swearwords or said something dangerous I bet you’d tell someone. If it had a spelling mistake in it would you keep it to yourself? Because that would erode the user’s confidence in your product from the very first time they interacted with it. And it’s so easy to take a cursory look at - provided you don’t have to write down the results every single time, of course, but that’s the curse of explicit formal test cases.

Isn’t blank password and invalid password processed in the same way?

If you don’t know then perhaps you should check to see if they both work. Or perhaps you could dig into the code (or ask someone else to) to see how they are processed so that you can better judge the risk. Even if the code treats them identically, and the code that runs the code treats them identically maybe the database doesn’t. Maybe some other third party system doesn’t.

I mean: is the scenario that we are unable to login with valid username and invalid password, but able to login with valid username and blank password even possibile at the same time?

Firstly, yes. Perhaps it’s possible to create a user with a blank password, or update a user to have a blank password, or edit the database to have a blank password, thus making a blank password a “valid” one.

Secondly even given the assumption that they are mutually exclusive scenarios that doesn’t necessarily mean that they are processed identically. Perhaps the code that protects against things like SQL injection attacks isn’t prepared to receive blank inputs. Your system should probably be using prepared statements, so when that data is fed to the DB how will it react? Does the prepared statement data code work with blank inputs? Maybe it’s some DRY code somewhere that interprets data types on the fly for some ungodly reason and it passes it as some kind of null value. Or, heaven help us all, you have some flimsy copy-pasted SQL injection filtering code full of holes that falls over. Your test results could help trigger an investigation to find out all of this kind of thing and potentially expose enormous design flaws. Or playing with the ideas could give you a lot to think about with the login page. You could look at the DB and see what you can do with it, and what you can get away with.

The important thing here is, I think, that you’re splitting up the cases into these equivalence sets that are rife with assumptions and abstraction leaks and treating them with much more respect than they deserve. Making groups of ideas like this is useful and important, but you must remember to treat them with much humility. For a start there’s everything you don’t know or haven’t thought of. Also everything about what you say might be true, but when you turn that into action (only checking one of them) you fail to find something important because you limit your actions and thought. You also never think deeper about the product or get feedback from it directly that makes you think of other problems that you might want to explore. Luckily you’re already doing this because here you are on a forum questioning your partitions, for which you should be proud, but I think it’s worth stating explicitly because it’s a useful way to think.

Finally I’ll say don’t worry about being confused. If you go through a career in testing without ever feeling confused you are terrible at your job. Ask the questions, try to find some clarity, and when you come to the overwhelming realisation of the sheer complexity of even simple computer programs and can wade in with tools, experience and knowledge and cry like three times maximum then you will know you have transcended the comfort of thinking in assumptions into the glorious world of thinking in assumptions but knowing that so you’re more careful.


Woah! This opens the door to account enumeration and a subsequent phishing attack. Always leave the error message ambiguous (i.e. “The username or password entered is invalid”).

Coming back to the original question, an invalid password and a blank one are technically different, and a blank password might reveal a logic flaw. Best to try everything you can think of just in case.

Also, if you know the devs have written their own login page, it’s worth trying the valid password from one account with the valid username of another, and also asking them what is actually stored: the password or just the password hash. :wink:


Guys, thank You for these great answers. I really appreciate it. It has cleared my doubts.

Just to clarify one more thing - if we want to verify error validation messages - can we use that valid/invalid set of test cases, and just add appropriate error message in expected result field? This would lower number of test cases but I am not sure if it is recommended?


TC: verify that user is unable to login with invalid username and password.
TC : verify that user is unable to login with invalid username and password and is presented with appropriate error message.

Expected result for these two: unsuccesful login and user presented with ‘invalid username or password’ validation message.

Or we have to cover it in another separated test case like:

TC: verify that user is presented with ‘invalid email or password’ validation message after entering invalid email and password’

Expected result: user presented with ‘invalid username or password’ validation message.


As @kinofrost said: stop bothering with garrulous test cases. Just make a list of things you to test and then test it. When you do something in the software watch out for everything you can.

Make a table with:

  • columns: “case”, “no loging possible (true/false)”, “error message”
  • rows (content of case): the different scenarios of invalid logins (one or both wrong or empty)

For god sake: include it into the existing TC, don’t create a new one for every detail.


Yes you can because unless you’re in some mandated environment or something there’s no rules! If they’re your test cases you can do whatever you like to them. You can even have a test case that says something like “Explore the login function being sure to check both valid and invalid login credentials and look for sensible error messages”. You can put everything into existing cases, you can use new cases or you can create a charter (similar to the one above) that covers all of those cases and update it with new ideas as they come to you. Test cases are just one tool out of many you can use, and your tools should always work for you more than you work for them.

Here’s an old article that began the idea of Session-Based Test Management, a way to use charters for more useful and engaging testing that helps you to cover more ideas faster Session-Based Test Management - Satisfice, Inc..

1 Like

Empty is definitely a valid thing to separately test.

Maby applications will have a separate code path to block/handle an empty password, possibly provided by the framework.

If the system is trying to log in with a blank password, then does it actually do the password check? Depending on if an empty string or null value is passed, you could hit some nasty behaviour. Finally if the system is encypting / encoding your password, does that work with a blank string?

1 Like

I’d be inclined to call it a bug if a login page allows you to submit a request with a blank password field - really, there should be page validation to throw up a warning if either or both of the username and password boxes are empty. Considering things from an API testing perspective, no credentials and invalid credentials are two separate cases, though both should see the API returning a 401; it’s rather like being refused access to a building when you have no security pass, and being refused entry when you have a security pass for a different building.


Story time:
years ago a consultant would contact a company about a security problem with the network software. It was possible to log in as a system administrator on the network. This basically means: I can do everything with all your files on your network.

During a small demonstration a program would access the network software. This program misused two bugs:

  • it was possible to enter an invalid password forever.
  • once in a while the program would accept an invalid password.

The default username of the system administrator was known, The used password was an empty string.