What testing terms are still causing confusion within the community?

🙋 Questions
,
1

I shared this question over on Twitter and LinkedIn, it just highlighted that there are some terms that come up repeatedly.

https://twitter.com/Travieso_nastya/status/1678810854312095749

https://twitter.com/helpermethod/status/1678791110196445184

Joep Schuurkes “all of them?”

Callum Akehurst-Ryan
“Exploratory testing - What do people understand this to mean?
Testing/Checking - Seen as a negative rather than a useful qualifier.
Tester/Engineer/QA - Job titles don’t have a consistent meaning.
Tester vs QA - What’re people understanding these to mean?
SDET - Does this just mean automator, toolsmith or a programmer?”

Do you have any you’d add? Or can you sympathise?

2 Likes
2

+1 to ‘all of them’

I don’t see it as just one testing community, but hundreds of them.
And just like states, religion, politics, and culture, the belief systems are partially shared within some of them.

6 Likes
3

I agree with “all of them”.

Part of the issue is that even if all the various testing communities were to reach a consensus on what the various terms mean, our employers would still use the terms they were comfortable with.

It’s one of the reasons I try not to get too caught up in the terminology wars.

Even basic English is used differently enough between the countries that speak English as a first language to cause communication problems: why would it be different in the testing communities? (as an example: in meetings/debates etc. the meaning of “to table” a topic is opposite between the UK and the USA - in one, it means taking the topic out of active discussion, in the other, bringing it into active discussion)

4 Likes
4

Are the terms confusing, or do we simply disagree about what they infer? Is a difference of terms a confusion, or a convergence of meaning with two different symbols? Is the meaning of “automation” Selenium suites, or the replacement of human thought and action with machines - and is the confusion about the former, the latter, or which one out of the two it is?

I think that people have different understandings and interpretations of the same terms based on their own prior experience, understanding, beliefs and worldview. I’ve discussed the meaning behind the words I use to many people, and it always ends up that I have to explain my perspective to make that work. “Exploratory testing” is a tautology, but it’s still a useful term - but I sometimes have to explain its history and evolution to make my perspective understood about the limitations of automation or the need for human-first testing and strategy building.

It’s important to know what the differences are and to try to understand when it matters. I wrote a little bit about this before, but I won’t go into it here, that wasn’t a happy time for me.

I think “automation” is a term that harms the testing industry when it’s used. I believe that it doesn’t accurately describe what it does and perpetuates the idea that people who do testing are unskilled and replaceable. James Bach gave a talk in which he brought up the example of “autocode” for compiling, and how developers refused the suggestion that their job could be seen as automatable (and with AI coming in I predict they will do so again soon). Testers let ourselves be dictated to by people whose roles can contradict our own, and I’d like it if we took that power back so we can get busy helping instead of performing nonsense under orders.

I how I also know that “automation” is a memetic victor, and that makes it as close to an industry standard term as we tend to see. So if someone wants to say “automation”, I think we both have a reasonably good idea of what that is made of, who does it, what kind of tools are involved, and so on. We know what it’s made of. We are more likely to disagree about what it can do, its value, its weaknesses and so on.

So are people confused, or do terms have different affordances and different implications for different people? And it is the terms that are causing confusion, or the lack of understanding of scientific philosophy, epistemology and information theory that is causing the confusion and the terms just lay unsteady on the gaps in our knowledge?

1 Like
5

I don’t think the terms are confusing.

People can deduce 80% of the meaning from the terminology but it’s the remaining 20% that can cause confusion.

As an example above, Defect vs Bug, I think we can agree that both terms mean a behaviour that is not expected (or undesired) in the software.
However, in my immediate org, defect is reserved for features that are in progress; while bug is for everything else (except escalation).

I agree with katepaulk. Don’t get caught up in terminology wars. If necessary, just clarify or ask for clarification.

1 Like
6

I find that we lack standard terms for different types of testing. What I think of when I say “integration testing” is different from what 10 different other people think of. “Component testing”, “system testing”, “workflow testing”, I’ve found other people have way different definitions than my own. I’ve tried to stick to the lexicon Gerard Meszaros came up with in his XUnit Testing book years ago (his terms apply to more than only unit testing). But most people don’t seem to have adopted a standard.

Another thing I see used incorrectly - people use “mocking” when they really mean “test doubles”. Mocks are just one type of test doubles. There are also fakes and stubs. Again, this is something I learned from Gerard’s books.

3 Likes
7

Are they wars of terminology, or of ideology?

If we’re going to talk about the words as seen or heard, as in the text or uttered noises, then they matter very little indeed. I agree with you that the terms don’t appear to be confusing. If we’re going to talk about what those words infer to people, then maybe we have something to talk about. All else being equal I think terminology doesn’t matter much. So we should examine what isn’t equal, and if that matters more.

I wrote a bunch about the dangers in terminology and expression here, which I think spells it out.

Another point that I don’t think I’ve made before is how words make people feel. I never say that I break software, not just because I think it’s incorrect, but because it puts me at odds with the people designing and building it.

So I agree that terms aren’t generally confusing, but terms can be misleading (“automation”) or emotive (“break software”) or otherwise charged with meaning beyond their intent. The swastika is an ancient religious and cultural symbol used even today as a symbol of spirituality, but it’s still banned in Germany.

8

Integration testing

Unit testing

End to end testing

I hate all of these terms. You can quite easily make an argument that any of the above tests is one of the two others. They lack precision.

Also TDD/BDD but with those it’s more because the popular conception doesnt match what they actually are.

2 Likes
9

+1 all of them

Language used during a project or testing phase should be established during a ways of working meeting - not a one way street where the language gets dictated to you - you’re also telling them what you mean by the terms you’ll use.

3 Likes
10

I know there are some who dislike the ISTQB courses, but I do think that the ISTQB glossary is a useful tool for providing standard definitions of terms which shouldn’t cause confusion.

Eg. Exploratory Testing
An approach to testing in which the testers dynamically design and execute tests based on their knowledge, exploration of the test item and the results of previous tests.

Although this doesn’t work for all terms and phrases.

2 Likes
11

Starting with a blog article from myself others added some more:

image
image850×223 60.3 KB

Any combination from this can (at least) either refer to automation code or the typical test (case) management format with actions and expected results.

6 terms for at least 2 different things (or a spectrum?) which people use pretty interchangeable !

+1 to ‘all of them’

1 Like
12

Oh boi… we can talk hours and days about this topic. Really great replies in this thread, I’ve also read @sebastian_solidwork’s Ambigous phrases - lots of gems there too.

I could reply to most posts but I’ll try to summarize my thoughts all in one go:

  • yes, ALL OF THEM can cause confusion. Depending on previous experience of both a person and company environment
  • on a new project, a developer introduced me to a client with “Here’s our Q&A”… Q AND A!!!
  • regardless if we refer to us as one testing community or hundreds of them, I think we still need some standardisation or else we have Wild West of terminology (and we do have Wild West…)
  • I would LOVE for ISTQB glossary - or any glossary for that matter, become an actual standard people can refer to. What I mean by that is if someone refers to smoke test with something other than it says in the glossary, that someone should be corrected to use more appropriate term. Problem is, ISTQB itself is not very respected in the testing community - I don’t want to get into heated debate here but that is my observation based on few years of experience
  • why is an agreement of having the same terminology beneficial? To make an exaggerated example, you do not see developers referring to a “git branch” as “git limb”, or “deploy” as an “upload”. It is simply well known and undisputed what a git branch, deploy or a download is (and git limb does not exist). You cannot (and must not) mix those terms.
  • in the end, yes it comes down to perspective. It almost always happen that I have to explain my perspective, as @kinofrost was saying, to fellow developers, QAs and the like, to be explicit of what I mean and what they mean. But why do that all…the…time… if we can have one unified terminology?
  • I fear that in the end, we the testing community, are also to blame for this confusion. Why? Because we invent new testing terms all the time! There is simply soooooo many terms that my head hurts. Just by simple google search I found out a page with these 105 types and many more exist. That’s just crazy.
  • I admit I never heard of “Test Double” until I’ve read this thread.

We all know this saying:

No new Javascript framework were created while reading this article.

I’d add my own two cents:

1 new Testing term was invented while reading this article: git limb.

2 Likes
13

Who controls the unified terminology? Because whoever controls the spice controls the universe.

Which business contexts, development methodologies, social structures, countries, languages and cultures do we deny?

I don’t think it’s going to matter.

2 Likes
14

Even if no everlasting unified terminology is possible I have the general impression that developers have way less ambiguity.
I see it as spectrum and developers are more away from ambiguity than testers.
We could surely improve some things.

15

Even if no everlasting unified terminology is possible

Is it wanted? I don’t want one. I think that it would stifle creativity, diversity and innovation, and it would be exclusionary, overly censorious and, I have to say, a little totalitarian.

developers have way less ambiguity

They don’t do the same job. Testing is unbounded, infinite, social, and chaotic more so than code writing. Code (and to a lesser extend its design) has to be standardised because it’s about properly controlling a very picky, deterministic machine (or talking about it or teaching how to do it, etc). Testing is about the influence of context, open-ended risks, evaluating the perceived quality of others, responsible communication, all in an infinite search space.

We also have different problems. We are struggling to be recognised as valuable, and we live in a world where the keys to the most recognised glossaries are held by testing-free development methodology producers and factory school certification printers. I’m happy to be able to use a different one - insistent, in fact.

We could surely improve some things.

What things would you like to improve?

I would like to improve peoples understanding of “automation”. I don’t think that’s a good term because it’s inaccurate - testing cannot be automated, and we shouldn’t advertise that it is. My goal isn’t a change to the terminology (although I think that would speed up the real goal), but to offer the change to raise awareness of everything the term infers, and the weaknesses and limitation of automation that go unseen, and help to devalue testing and testers. I don’t really mind about the words, it’s the shared meaning that matters - and the meaning will differ in some way between contexts no matter what I do.

Do we use the ISTQB definition of “test”? “A set of one or more test cases.” If not, then what is the regulating body for the international standard of terminology for the definition of “test”? If we don’t use one then who decides what the one true definition is? And if there are multiple definitions who gets to use the word “test”, and who has to come up with new terms? And who tracks who uses what terms? And slowly we work our way down to where we already are. I don’t think testing takes particularly well to standardisation. I think that its terminologies rely on contextual descriptivism, and the only way to guide that well is better thinking in the world of testing. I cannot be context driven and dogmatically prescriptive. I cannot tell people that the value of anything they do is defined by their context, then provide a single list of terms for every context. There are no best practices, just good practices in context.

16

Interesting choice of a proposition.

The people that coined, defined, and promoted it had:

So why would ISTQB be an authority regarding it or any other term where they’ve applied similar tactics?

1 Like
17

That is a good point, and I would say most (in fact, probably all) software testing terms would have existed before ISTQB came along. They certainly didn’t invent a lot of the terms and definitions (or at least I hope they didn’t). They just took all the available information, and used it to formalize a syllabus and glossary. My main reason why I refer to the ISTQB glossary is because it contains the majority of definitions required in one place.

I do believe that ISTQB deserve more repect that it is given. Someone (I can’t recall who, but it during a conference talk) pointed out that testing as a profession was taken more seriously when the ISTQB courses and qualifications were developed. I don’t know if this is true or not as my software testing career started after ISTQB started.

ISTQB is far from perfect. The courses are purely knowledge based rather than about learned experience (hate those multiple choice questions). Good for those just starting out (I took the foundation course when I was new to testing and got a lot out of it) but not for those with more experience. It definitely needs updating.

18

From the ISTQB glossary:

test: A set of one or more test cases

That’s enough for me to ignore it.

Also, on the topic of exploratory testing, the ISTQB glossary says:

exploratory testing: An approach to testing in which the testers dynamically design and execute tests based on their knowledge, exploration of the test item and the results of previous tests.

Which, for the namespace of its origin, is incorrect: Exploratory Testing 3.0 - Satisfice, Inc..

And this seems to be true historically, also: Exploratory Testing is not "Experienced-Based" Testing - Satisfice, Inc.

And the entry contains “References: After ISO 29119-1”, which is a standard rejected by people I respect for worthy reasons.

I suppose that’s a good example of my argument against official glossaries. There’s a large, educated community of people who reject many of the ideas taught by the ISTQB, and therefore the reflection of that in their terminology. I believe their definition of “test” is factually and ethically wrong. Consensus on terminology becomes impossible as long as there is a diverse marketplace of ideas.

19

Oof, that is terrible. Not sure what to say to that, to say it needs improving is an understatement.

20

API testing.
Why do we use that term for web services 99% of the time, when 20 years ago before web services were how apps got built, API testing usually meant local method calls. But in fact API tests meant OLE or COM/DCOM or more scarily meant knowing x86 calling conventions - Wikipedia and PASCAL-style and STDCALL calling conventions .

1 Like