What’s the difference between a human test and an automated check?

🗄️ Archive
1

We’re not entirely sure whether the discussion between human and automation testing will ever end.

But we do love to be here to give voice to what testers feel about the topic and how they personally define it - what is the difference between a human test and an automated check? How do you feel about it? What should we be saying when people ask this question?

We put the question to the community (via Twitter, LinkedIn and Slack) and here we have curated many of the responses.

What’s the difference between a human test and an automated check?

Feel free to add your thoughts in!

Speed. Human test are more visual and can find new defects. Humans provide better coverage. - Ashish Jain

Stability, an automated check is not able to work around issues on the fly. - globalworming

There are (and will be) a lot of opinions. But I would say at least one difference is:

  • Humans have decision power/critical thinking and can understand if in some edge behavior the SW is still operating between the required thresholds.
  • Automated checks/tests, follow a script that will only validate what is programmed (automated). The automation can be very strick and contemplate a lot of behaviors (including some edge behavior) but will not include all and will not have the critical thinking to fully understand what is happening and judge if it is acceptable. - Hugo Fernandes

A human is capable of tangential inference. An automated check is only capable of assessing a particular statement in, generally, a boolean pass/fail context. - Joe Triccas

Automated checks relate to asserting explicit expectations.
Human tests often relate to expectations (but with an additional eye for checking implicit and tacit expectations too), but importantly it more commonly relates to unexpectations - investigating risks, unknowns, variables, assumptions, etc - Dan Ashby

Automated checks will always follow the same steps and automated check will always check what it is told to check. Nothing more and nothing less. - Karulka

A human can write an automated check. From my physics point of view, the difference is running an experiment to find new behaviour vs. running repeated measurements in order to support (or rather invalidate) a heuristic/model/theory. - Stephan Kämper

The difference is in the capacity to make a judgment in terms of quality, outcomes, purpose, meaning. But now that you ask, I think we’re misusing “check” as the word comes from shah, king so etymologically it’s a decision made with both judgment and power. - Anders Dinsen

Automated checks are subsets of human tests. Different ways humans execute tests and analyse the results, automated tests fall within this. Not the other way round - Richard Forjoe

Many things, but can your automation show something that makes you feel or is/is not intuitive? - Chris Armstrong

The one needs all brains upfront. The other needs all brains upfront, during and after. - Patrick Prill

Multiplicity - Ady Stokes

Let’s say you can’t remember if you’ve turned the stove off when you left the house this morning. You send both a human and an automation test to check. The human comes back screaming your house is on fire while the automation just deadpans you with a ‘No’. - Ursa Kacar

Human tests are exploration, automated checks are sentry duty. A human test can make an observation and branch from that. Learning and adapting along the way. Automated checks are more for patrolling the same area and calling out things that have been predicted to go wrong. - John Dorlus

Exactly that. Automated checks are checking what they were written to check. Human tests are testing the system, testing the limits, testing the happy/negative paths, testing accessibility, testing performance, testing usability, testing their patience and those of the dev team - theqadiva

As of now Human can perform functional exploratory testing but automated check only test you mentioned in test suit,but not sure in AI ML world it may perform exploratory too. - theqaguy

Human tests can check for functionality, usability, accessibility, user experience, etc. whereas automation is mosty used for functional checks.
Automation is a complement for human testing. - Venugopal Botla

A check only looks for what you tell it to look for.

A human doesn’t. - Stu / Wildtests

It’s the difference between you toasting a bread in a pan and the toaster doing it for you. - harryhkwtd

User experience (as opposed to only looking at whether something is outright broken or not). - MindfulQA

They are not separate, human can automate some tests to accelerate results.
So automated tests are also implemented by human. - AwezanSaeed

A computer can ONLY check. A human CANNOT only check. A human will notice stuff beyond the bounds of check. A computer cannot. - Richard Paterson

The human can use initiative and curiosity, the automated check can’t. - Adrian Roberts

What’s the difference between a human developer and a automated developer? - arquillos

One of them does not enjoy the landscape. - Ivan Rodriguez

The ability to think about it. - Alex Wotton

Sometimes, hours, so I’ve been told. - Stephen Brown

The automated check is more prone to error :sunglasses: - Karl Lobato

Which one can uncover that the system works perfectly but isn’t what the user needs? - Liz Read

While Time and money seem to be the big differentiators , I think the biggest of them all is ‘freedom’. - Brijesh Deb

Testing conducted by human involves sapience during the execution of tests. Automated execution does not. But testing is way more than just execution. It involves interaction with other people, planning, preparations, choosing approaches, adapting to context, analysing results, acting on information, maintenance on environments, data, etc. etc. All those involve sapience, even in contexts that have heavy emphasis on automation. Time saved in automated execution is often lost in test suite upkeep, configuration management, keeping up with dependencies, etc. So human testing and automated checking are both quite time consuming after all. No real difference there. - Sami Söderblom

Automated checks validate known criteria only. Human tests go beyond known criteria into edge cases. - Dave Taddei

One is faster to check knowns and the other is slower but can explore/uncover unknowns - Luke Liu

One needs human input and the other doesn’t. That’s the main one. We can talk about what’s better in different context, gaps in one and the other and so on…but the main difference is simply that. - Patricio M.

The answer equates to creativity Vs logic. One is limitless and exhaustive while other is limited but symmetric. - Raza Sohail

Consistency (which could be good or bad); repeatability ; time ; cost ; depth ; morale ; responsibility. But mostly I would say context. - Luke Robbertse

Automation Test: Can perform with great speed compared to human but cannot perform ‘Out of the box’ until coded… - Prasoon Prakash Shukla

The critical eye - Arif Taseer Asif

3 Likes
2

[What’s the difference between a human test and an automated check?]

It’s like asking what’s the difference between a human and an animal and then comparing them by varying criteria, without defining the human, the animal or their contexts:

  • Speed
  • Survival instinct
  • Loyalty
  • Perseverance
  • Capacity to provide food and shelter
  • How much food and water each need
  • How much waste each creates.
    etc…

So on the question, I’d say it depends on what kind of human you’re talking about, what he knows about testing, what he tests, what’s the budget, what’s the scope, what’s the automated checking’s purpose, what resources are available, what timeframe is given, what expectations there are…

2 Likes
3

I am looking a bit at AI at the moment and one element I read about was that the coding for big blue which beat kasparov at chess is actually a fairly simple view of AI and that decades later what turned out to be a much harder challenge for AI was actually picking up the chess piece and moving it around the table.

For a human the latter is for most simple and the former again for most more complex, hundreds of thousands of years of evolution results in a full on sensory buffet in almost everything we do.

AI is advancing a lot and even if it could be viewed as autonomous learning, decision making and adaptability to varying degrees of uncertainty it is generally still has a very narrow field of vision, a few tasks or things to do.

An automated check is an even simpler version of this with a focused single factor assertion.

A human test on the other hand has that evolved full on sensory buffet that has a broad view of things even if it is considering the same single assertion.

Each have their strengths vs weaknesses, chess vs picking up and moving a chess piece is a simple example but perhaps the narrow vs broad comparison has some merit.

1 Like
4

It depends on the tester and the circumsatnces involved.
Some test houses are no better than automated tests.
Identify Test Case,
Identify entry parameters
Identify exected return.
If there are any exceptions, they are recorded as a fault.
Will a test house tester identify a spelling mistake, similarly, would an automated test set identify the same issue.
Automated tests have no memory. They are repeatable and operate at the vanilla common path level for most of us.
Testers can have a memory, and most would call on that to identify differences from one run to the next.

5

Common sense, heuristic and to know when to stop

6

Humans are really good at recognizing patterns, including when something that should be somewhere isn’t.
Humans are really good at making judgments on the fly.

Automated tests have no judgment beyond what’s programmed into them (I’m not talking AI here)
Automated tests are really good at doing the same thing the same way over and over again and not getting bored.

My short-short version is that automated tests are good things to use to speed up time-consuming, boring things like regression tests or lengthy configuration setups. Humans are good for figuring out that the UI is a really bad idea because it’s confusing or the layout doesn’t make sense.

8

When it comes to software testing, it can be done in both ways: Manually and via Automation.
In software quality assurance companies both types of engineers can be found and both are useful.

Where automation saves time in testing the same thing every day, the manual tester can do much better Adhoc testing.
Here are some key differences of uses of both :

Manual :

  1. Manual testers can perform random testing.
  2. GUI testing can be performed in a much better way.
  3. Cost-effective for small projects as the investment is less.
  4. Programming skills are not required.
  5. This testing is more preferable if Application Under Test changes frequently.
  6. Manual Testing is suitable for testings like Adhoc, Exploratory, and Usability.

Automation :

  1. Automation testing is faster than manual testing.
  2. There is no testing fatigue.
  3. No chances of human error.
  4. Performance testing is possible such as Load Testing, Stress Testing, Spike Testing, etc.
  5. Tests can be executed in parallel.
2 Likes
9

Human testers can test what happens when the human user does stuff wrong. Automated tests only check that what is the right way of doing stuff works properly.

10

We can think of checking as a part of the activity of testing.
The checking can have more or less aid of certain tools (e.g. general computers).

image

1 Like