- Why not consider those things earlier
Depends on the things and their granularity, but basically because we don’t have access to enough information to make good decisions or come to useful conclusions. We can think about how much time we have, but if something big changes, we lose a client, we gain a client, the market shifts, half the staff get ill, then no amount of thinking about the time we had early on will change the new reality. And so it is for all things. It’s hard to know how much effort something will take without investigating it first. Development effort is absolutely no help either, as something that’s easy to make can be hard to test properly, like a search bar or an important new input field. It may be revealed that a client finds a particular thing very important to them.
- What is “earlier” - when is the test planning done
I’d venture to suggest that test planning is never done, it just continuously evolves as we get new information. As the project and product twists and turns, changes, and of course reveals itself to us as we investigate it, our understanding of logistics, resources, strategy, quality, techniques and environment is going to change, and so what we actually choose to do is going to change. The alternative is to stick more to a plan we started with even when that plan starts to not make sense, and if you want my argument against the waste and tendency to follow instructions over reality that premature and over-formalisation generates, there it is.
As for when initial, formal, written, test planning is done that feels like it’s going to be very contextual, and based on why someone feels that high-formality documentation is actually useful in the real world. Given a concrete example I suppose maybe that would be easier to talk about, but I can’t think of one right now. I guess planning a plan is a thing. What is the plan for? Is it to give someone else something they want, or a tool you’re going to use to help your testing? Why are we making one, and therefore what should it contain?
Anyone who’s done a session and seen 7 new sessions drop out of it knows that exploration breeds exploration, so what is so resistant to exploratory revelation that we can talk about it without booting the product? Or perhaps more importantly that we should? As always, it depends.
- What does the process of test planning look like for you; does it being formalised prevent you from having better conversations / involving more people?
That is a very good set of questions.
For me test planning is just consideration. It happens in the minds of a tester, up until that information needs to be recorded to be shared or remembered. When it’s recorded the whole plan doesn’t go in the document. It’s just a representation of the thoughts someone has about a situation. It probably has some generalised thoughts about test coverage, risk ideas, plans to mitigate those risks, testability issues, very broad schedules and the like - whatever feels like a good idea. I’d also say that not everything in a test plan will survive contact with air, and that’s okay. We need space to have a worry and then watch it evaporate if it needs to.
A test plan could remain all in my head. It could be scribbled on a white board for the team. It might be a lengthy digital document.
does it being formalised prevent you from having better conversations / involving more people?
I think that formalisation can lead to fetishization. Why do we need to do this thing? Because of The Document! Can’t we just… No! The Document! It’s a convenient and easy replacement for the hardest thing testers have to do and one of the things companies often hate to do - deal with reality. We will build any tool and invest any amount into not dealing with the fact that stuff is hard to learn and it requires people thinking about it. It depends a lot on what The Document contains, how it is written, what it contains, who wrote it and the general attitude of people towards it including management.
It also can be made to look and feel a lot more intelligent and thoughtful than it actually is. Formalisation often comes packaged with such an official and comfy feeling to it because it’s a communication - a performative one, at least in part. If I say “I play with the product and learn things” that’s the same as “product exploration with the intent to reveal product issues and information pertinent to the test effort”, but the second one is the one I’d pick when I want people to think I’m clever and capable. It doesn’t offer more to reality, just to the way people perceive what I’m doing. And this mechanism works to make nonsense look clever and capable, too, especially when it’s not questioned, and questioning other people’s public documents is effortful and risky in a way a conversation is not.
If a formalised document is flexible, living or disposable, then great. I think that’s much more in line with how the exploratory nature of all testing and learning works. It does mean that the more formalised it is the more maintenance is required and the more things in it can become wrong or outdated. The more granular the formality the higher the maintenance cost, it’s just explicit, detailed test cases writ large. If we know the document’s maintenance is being outpaced by reality then it loses its status as a useful oracle.
From reading the other responses, it makes sense to me for a formalised test plan to be more of a tool that triggers a set of good practices, rather than a goal to have a document.
This feels very true to me, too. As formal as it must be to achieve useful things for its cost. No best practice, only good practice in context.
Edit: I’m suffering brain fog from my condition, so if this makes no sense or wanders you have my apologies. I’d blame the lengthiness on this too, but I always do that.