First Iâll just say if you stop looking at testing as a series of lines of checks and more as an exploration based on your understanding of the product guided by risk you can achieve more with much less writing, reduce the cost of upkeep, avoid going over old ground again and again, plus itâs more engaging and fun and you might find some really good problems that the test cases donât account for. Youâll find that you can do more interesting test cases if you donât write them down in that way. Move towards a checklist of suggestions rather than a case-by-case basis. This deformalisation will let you do more testing and less writing with more freedom to ask interesting questions of the product. What you actually consider when you test is, letâs face it, not determined by whatâs on the test case list but with your real-life interactions with the product.
You pose a lot of interesting hypotheses. If all of our hypotheses about the product were true then thereâd never be any bugs in it - after all the product is developed to work and one big hypothesis is that everythingâs okay because we made it to be okay. This is clearly not true, so we must learn to distrust our hypotheses and test them to turn them into more robust theories of the product.
To the specifics:
Isnât it just a repetition and unnecessary additional test case when considering blank password, if we already consider the invalid one?
Maybe. Perhaps some part of the code that expected to receive an input simply falls over when thereâs no input provided. Perhaps it is supposed to display a different messageâŚ
( We donât consider error validation messages)
Donât you? Because if it was full of swearwords or said something dangerous I bet youâd tell someone. If it had a spelling mistake in it would you keep it to yourself? Because that would erode the userâs confidence in your product from the very first time they interacted with it. And itâs so easy to take a cursory look at - provided you donât have to write down the results every single time, of course, but thatâs the curse of explicit formal test cases.
Isnât blank password and invalid password processed in the same way?
If you donât know then perhaps you should check to see if they both work. Or perhaps you could dig into the code (or ask someone else to) to see how they are processed so that you can better judge the risk. Even if the code treats them identically, and the code that runs the code treats them identically maybe the database doesnât. Maybe some other third party system doesnât.
I mean: is the scenario that we are unable to login with valid username and invalid password, but able to login with valid username and blank password even possibile at the same time?
Firstly, yes. Perhaps itâs possible to create a user with a blank password, or update a user to have a blank password, or edit the database to have a blank password, thus making a blank password a âvalidâ one.
Secondly even given the assumption that they are mutually exclusive scenarios that doesnât necessarily mean that they are processed identically. Perhaps the code that protects against things like SQL injection attacks isnât prepared to receive blank inputs. Your system should probably be using prepared statements, so when that data is fed to the DB how will it react? Does the prepared statement data code work with blank inputs? Maybe itâs some DRY code somewhere that interprets data types on the fly for some ungodly reason and it passes it as some kind of null value. Or, heaven help us all, you have some flimsy copy-pasted SQL injection filtering code full of holes that falls over. Your test results could help trigger an investigation to find out all of this kind of thing and potentially expose enormous design flaws. Or playing with the ideas could give you a lot to think about with the login page. You could look at the DB and see what you can do with it, and what you can get away with.
The important thing here is, I think, that youâre splitting up the cases into these equivalence sets that are rife with assumptions and abstraction leaks and treating them with much more respect than they deserve. Making groups of ideas like this is useful and important, but you must remember to treat them with much humility. For a start thereâs everything you donât know or havenât thought of. Also everything about what you say might be true, but when you turn that into action (only checking one of them) you fail to find something important because you limit your actions and thought. You also never think deeper about the product or get feedback from it directly that makes you think of other problems that you might want to explore. Luckily youâre already doing this because here you are on a forum questioning your partitions, for which you should be proud, but I think itâs worth stating explicitly because itâs a useful way to think.
Finally Iâll say donât worry about being confused. If you go through a career in testing without ever feeling confused you are terrible at your job. Ask the questions, try to find some clarity, and when you come to the overwhelming realisation of the sheer complexity of even simple computer programs and can wade in with tools, experience and knowledge and cry like three times maximum then you will know you have transcended the comfort of thinking in assumptions into the glorious world of thinking in assumptions but knowing that so youâre more careful.
I hope it doesnât sound to harsh? XD