During last week’s This Week in Testing “critical thinking” got another mention.
It was concerning AI and that as testers we have to remain critical thinkers even when we rely on AI tooling to support our efforts.
There’s a lot of good stuff about critical thinking on the MoT site yet I wondered, where else do you learn about critical thinking? Are other industries growing it as a skill? How might we learn from them?
Also, how do you practice critical thinking for someone new to it?
For me its one of the toughest areas to teach.
Culture and education are likely the most important aspect.
Your four year old asking why?, why?, why? for days on end is generally a very good thing on this front.
Not all cultures support that model though and I meet many people that have more of a do what your told in the way that you are told to do it format.
Education is similar, a UK degree for example its very hard to avoid critical thinking particularly on the science side of things.
Comparing that with some degrees in Asia where I am there are definitely some cases where its been absent from the whole education system even at degree level. That’s not a broad brush sweep but its evident enough that hierarchical cultures in my view potentially limit it as an every day learning activity.
I believe though it can be taught late but at the same time be aware of the cultural aspects that may make it quite a mind model shift for many.
If you have had some of the cultural aspects in your past I really suggest focus on joining an international company where you will be surrounded with a lot more people to learn from.
Testing is both an art and a science, that latter part is key and lots to learn from others in science and experiment based industries.
Let’s call this two things: understanding why we need critical thinking, and how to do it better. I think the former is important because without understanding why it’s so important it can seem like a hobby rather than about the human understanding of the universe, and our understanding of what’s actually going on in software.
Why It’s Important
To want to need critical thinking it’s important to know about the philosophy of science. How do we know what we think we know? Why can’t we just trust what we see and hear? Why is our own logical thinking not sufficient? When we understand how deeply flawed our natural understanding of the universe is, and how invented our perceptions are, then that gives us a state in which we can strive for greater understanding under the humility of our flaws as evolved ape creatures.
From there we grow science, the tool and method we use to get close to the truth despite ourselves. And we discover measurement, precision, falsification, biases, observation, inference, and epistemology. All the building blocks that go into understanding, learning, and testing.
My text book for philosophy of science was What Is This Thing Called Science?
Tool For Improvement
One tool I heard from a James Bach clip years ago was “Huh? Really? So? And?”. This is an oracle to help question anything. Words, pictures, product design, product behaviour, project, strategy, whatever you need it for.
| Question | Meaning |
|---|---|
| Huh? | Do I understand this? |
| Really? | Is what I understand true? |
| And? | Do I know the whole story? Is there more? |
| So? | Does the truth matter? How much? |
For example, someone says “defect density tells us if we can release the software or not”.
Huh? What do you mean by “defect density”? This is a reminder that we might not understand, and we need that understanding to employ or criticise something.
Really? Does defect density really tell us if we can release the software or not? This is a reminder that perhaps the facts of the matter are up for question. It may be false, or partly false.
And? Is there something else that we need to decide if the software is ready? Are there other meanings to infer here (e.g. defect density is actually useless, but part of a mandated process to comply with an industry standard).
So? It might tell us that, but there are more important things to look at. I don’t care about this metric because we’re on a contractual deadline that supersedes it right now. It may tell us that, but we don’t have to listen. Sir, this is a Wendy’s.
Give it a go on your next design doc.
I believe critical thinking is something that cannot be developed overnight. It is achievable through practice in the real world. The major issue of critical thinking is that its scope is very vast and one cannot learn from any source and gain a certificate and claim as critical thinking.
Yeah but there is one thing that one can try like practicing the critical thinking questions online and then reviewing the answers to see which answers were right and which were wrong, I’m not sure how much this will be helpful but at least it will give you an idea how strong is your critical thinking ability.
Also if someone is looking to improve their critical thinking then they should think beyond normal testing and pick up the task which requires such thinking.
For e.g. risk analysis - it is something where critical thinking is required, so testers should involve in such work which could help them in gaining expertise on critical thinking by working on live projects.
What does the phrase “critical thinking” mean? Lack of definition of terms is a stumbling block when we ponder any complex problem.
Believe it or not I’m writing something at the moment that may provide an approach.
That’s a very worthy question, and I have saved a summary from Dr Linda Elder, someone who has written many books on critical thinking:
Critical thinking is self-guided, self-disciplined thinking which attempts to reason at the highest level of quality in a fair-minded way. People who think critically consistently attempt to live rationally, reasonably, empathically. They are keenly aware of the inherently flawed nature of human thinking when left unchecked. They strive to diminish the power of their egocentric and sociocentric tendencies. They use the intellectual tools that critical thinking offers – concepts and principles that enable them to analyze, assess, and improve thinking. They work diligently to develop the intellectual virtues of intellectual integrity, intellectual humility, intellectual civility, intellectual empathy, intellectual sense of justice and confidence in reason. They realize that no matter how skilled they are as thinkers, they can always improve their reasoning abilities and they will at times fall prey to mistakes in reasoning, human irrationality, prejudices, biases, distortions, uncritically accepted social rules and taboos, self-interest, and vested interest. They strive to improve the world in whatever ways they can and contribute to a more rational, civilized society. At the same time, they recognize the complexities often inherent in doing so. They avoid thinking simplistically about complicated issues and strive to appropriately consider the rights and needs of relevant others. They recognize the complexities in developing as thinkers, and commit themselves to life-long practice toward self-improvement. They embody the Socratic principle: The unexamined life is not worth living , because they realize that many unexamined lives together result in an uncritical, unjust, dangerous world.
And one big reason that I like this summary is that it includes the social impacts of critical thinking and its absence, and shows our responsibilities when it comes to humility and truth. It’s a worthy introduction, but has plenty still to explain. What “high level of quality” means in regards to thinking is a difficult one to unpack, but rather than leap that mountain I think it’s much easier to climb one step at a time through the self-critical steps that led to the promotion of natural philosophy to science.
Edward Glaser writes:
The ability to think critically, as conceived in this volume, involves three things: ( 1 ) an attitude of being disposed to consider in a thoughtful way the problems and subjects that come within the range of one’s experiences, (2) knowledge of the methods of logical inquiry and reasoning, and (3) some skill in applying those methods. Critical thinking calls for a persistent effort to examine any belief or supposed form of knowledge in the light of the evidence that supports it and the further conclusions to which it tends. It also generally requires ability to recognize problems, to find workable means for meeting those problems, to gather and marshal pertinent information, to recognize unstated assumptions and values, to comprehend and use language with accuracy, clarity, and discrimination, to interpret data, to appraise evidence and evaluate arguments, to recognize the existence (or non-existence) of logical relationships between propositions, to draw warranted conclusions and generalizations, to put to test the conclusions and generalizations at which one arrives, to reconstruct one’s patterns of beliefs on the basis of wider experience, and to render accurate judgments about specific things and qualities in everyday life.
And here I appreciate not just the knowledge of the methods and the ability to apply them, but the motivation to do so. The will to do the difficult thing and chase what is true over what is easy and fold that into our understanding to guide our future attempts to understand. Which is what the exploratory nature of testing is all about, and at least to me somewhat explains the oft-used phrase “testing mindset”.
I’d like to return to the topic of driving critical thinking by understanding the need for it. I think that when a person realises just how wrong they are all the time then the idea of having to get down to details becomes important. When I taught testing I would show people things that illustrate the invented nature of reality, like their ocular blind spot, where the eye cannot detect anything where the optic nerve joins the eye, so the brain fills in the gap for you. Or how the brain re-colours surfaces based on your expectation of how dark it is. Learning about biases is not very good at stopping people from being biased, but it is a warning against simple thinking and shows the value of critical thinking and scientific method - the idea that we can make decisions based on reinforcing our existing beliefs, or on the first bit of information we hear, or that our memories change based on questions about those memories, and so on, shows how flimsy our understanding is and then the need to evolve a way to get at the truth through all that mess becomes very clear.
The reason I return to this now is that I think it’s much, much easier to understand what critical thinking means when someone understands why we need to do it at all. Its definition feels, to me, reliant on a certain understanding of its purpose, because when you understand our “human irrationality, prejudices, biases, distortions, uncritically accepted social rules and taboos, self-interest, and vested interest” then you understand that critical thinking is thinking that attempts to supress all of those things so that, when it matters, we can avoid failures in reasoning. In testing we seek to avoid failures in reasoning so that we can find problems that failures in reasoning put there in the first place.