I don’t write them. It’d be an exception if I did.
Words
First don’t worry about the terminology, everyone uses different terms. I don’t use “manual”, for example, because I figure all testing uses tools of all kinds, and it makes automated testing sound more special or capable than it is. But I also get what people are trying to say most of the time, so I can get a conversation done just fine.
Test Cases
Writing down test steps for someone else to perform is a very costly, tricky and error-prone activity. It’s where someone has a test strategy, so they write down what they think they might want to do to fulfil that strategy so that someone else can intepret what they said so they might understand what the author thought they might want to do to fulfil a strategy they don’t understand.
- Writing things down takes time
- Executing more steps takes more time
- Filling in each step takes more time
- Maintaining the cases takes time
- Every detail you include can be interpreted differently
- Every change made in the product has to be reflected in the details you add, or it’ll multiple people’s time to compare, decide and update them
- Focusing on details causes you to miss problems
- Following tedious instructions reduces engagement and attention
- If you follow instructions you cannot follow up on new ideas, or use what you’ve learned to create new cases, without going outside of the cases. You also can’t ignore what seems low value or outdated without failing to complete them.
Forcing yourself to repeat your steps reduces the chance of you finding anything new, so more details encourages people to deviate less and find less. Even if you believe that it’ll increase the integrity of the tests and make them more accurate there’s no guarantee that people will understand or interpret them the same way, or that the people that wrote it will still be working there.
Computers are for repeating tedious instructions, people are hired to think. You want to take advantage of their ability to learn quickly, see patterns, interpret their surroundings through an astoundingly complex set of heuristics and models to come to useful conclusions, all things that humans can both excel at and improve upon. Hiring people to be computers is wasteful, both in terms of the waste in extremely high cost, and the waste of the power and value of humanity.
My approach is:
- Never write down a case unless it becomes necessary. It’s costly and problematic.
- State what I want from a strategy, not what someone else should do to try to fulfil it
- Stay as high-level as I can, include detail when it’s necessary for clarification
- Link to resources
- Only note and report on what’s demanded of me and cannot negotiate, necessary for the current process, or useful in context
Complexity
So if we’re going with a higher-level, human-based, defocus-friendly, deformalized system then it will fall upon testers to properly investigate and learn your product. If your product, or any part of it, is complex then that complexity should be a red light going off in the tester’s mental dashboard. Complexity is a risk. A good tester will be looking for any way to make that complexity more approachable. Manuals, documentation, training, diagrams, models, maps, websites, comparable products, whatever. One way is to put testers in front of the product and have them engage with it. You want to have insight into what’s going on, so logging, distinct output, strong oracles, easy access to data, controlling and generating test data or anything else that can control states or affect configuration. Being involved early, to see the design and have a say in testability.
If I feel like I need low level steps or screenshots there are two questions I can ask:
- Why do I feel like I need these?
- Do they have to go into test cases?
Could you replace the steps with learning the product? A manual? Some training? A better or different strategy? Having better control over product state like save states or test data setups? Do the screenshots need to be in test cases, or can they exist in notes, knowledge bases, etc?
Are you using the test cases to help testing, or using them to tell people what to do? If I’m using the cases to be told how to use the product I could be much more effective, or effective at all, if I just learned to use the product. I can leverage my understanding of the product and its context to evaluate risk and look for problems much more effectively.
Eventually you should end up with someone who knows why they’re with enough skill and knowledge to decide how to go about testing so they don’t have to be told exactly what to do, and it will be much less crushing to the soul as a free bonus.
That makes all the formalisation unnecessary, everyone happier, everything cheaper, and you’ll find more problems, faster.