What’s the difference between a human test and an automated check?

We’re not entirely sure whether the discussion between human and automation testing will ever end.

But we do love to be here to give voice to what testers feel about the topic and how they personally define it - what is the difference between a human test and an automated check? How do you feel about it? What should we be saying when people ask this question?

We put the question to the community (via Twitter, LinkedIn and Slack) and here we have curated many of the responses.

What’s the difference between a human test and an automated check?

Feel free to add your thoughts in!


Speed. Human test are more visual and can find new defects. Humans provide better coverage. - Ashish Jain


Stability, an automated check is not able to work around issues on the fly. - globalworming


There are (and will be) a lot of opinions. But I would say at least one difference is:

  • Humans have decision power/critical thinking and can understand if in some edge behavior the SW is still operating between the required thresholds.
  • Automated checks/tests, follow a script that will only validate what is programmed (automated). The automation can be very strick and contemplate a lot of behaviors (including some edge behavior) but will not include all and will not have the critical thinking to fully understand what is happening and judge if it is acceptable. - Hugo Fernandes

A human is capable of tangential inference. An automated check is only capable of assessing a particular statement in, generally, a boolean pass/fail context. - Joe Triccas


Automated checks relate to asserting explicit expectations.
Human tests often relate to expectations (but with an additional eye for checking implicit and tacit expectations too), but importantly it more commonly relates to unexpectations - investigating risks, unknowns, variables, assumptions, etc - Dan Ashby


Automated checks will always follow the same steps and automated check will always check what it is told to check. Nothing more and nothing less. - Karulka


A human can write an automated check. From my physics point of view, the difference is running an experiment to find new behaviour vs. running repeated measurements in order to support (or rather invalidate) a heuristic/model/theory. - Stephan Kämper


The difference is in the capacity to make a judgment in terms of quality, outcomes, purpose, meaning. But now that you ask, I think we’re misusing “check” as the word comes from shah, king so etymologically it’s a decision made with both judgment and power. - Anders Dinsen


Automated checks are subsets of human tests. Different ways humans execute tests and analyse the results, automated tests fall within this. Not the other way round - Richard Forjoe


Many things, but can your automation show something that makes you feel or is/is not intuitive? - Chris Armstrong


The one needs all brains upfront. The other needs all brains upfront, during and after. - Patrick Prill


Multiplicity - Ady Stokes


Let’s say you can’t remember if you’ve turned the stove off when you left the house this morning. You send both a human and an automation test to check. The human comes back screaming your house is on fire while the automation just deadpans you with a ‘No’. - Ursa Kacar


Human tests are exploration, automated checks are sentry duty. A human test can make an observation and branch from that. Learning and adapting along the way. Automated checks are more for patrolling the same area and calling out things that have been predicted to go wrong. - John Dorlus


Exactly that. Automated checks are checking what they were written to check. Human tests are testing the system, testing the limits, testing the happy/negative paths, testing accessibility, testing performance, testing usability, testing their patience and those of the dev team - theqadiva


As of now Human can perform functional exploratory testing but automated check only test you mentioned in test suit,but not sure in AI ML world it may perform exploratory too. - theqaguy


Human tests can check for functionality, usability, accessibility, user experience, etc. whereas automation is mosty used for functional checks.
Automation is a complement for human testing. - Venugopal Botla


A check only looks for what you tell it to look for.

A human doesn’t. - Stu / Wildtests


It’s the difference between you toasting a bread in a pan and the toaster doing it for you. - harryhkwtd


User experience (as opposed to only looking at whether something is outright broken or not). - MindfulQA


They are not separate, human can automate some tests to accelerate results.
So automated tests are also implemented by human. - AwezanSaeed


A computer can ONLY check. A human CANNOT only check. A human will notice stuff beyond the bounds of check. A computer cannot. - Richard Patterson


The human can use initiative and curiosity, the automated check can’t. - Adrian Roberts


What’s the difference between a human developer and a automated developer? - arquillos


One of them does not enjoy the landscape. - Ivan Rodriguez


The ability to think about it. - Alex Wotton


Sometimes, hours, so I’ve been told. - Stephen Brown


The automated check is more prone to error :sunglasses: - Karl Lobato


Which one can uncover that the system works perfectly but isn’t what the user needs? - Liz Read


While Time and money seem to be the big differentiators , I think the biggest of them all is ‘freedom’. - Brijesh Deb


Testing conducted by human involves sapience during the execution of tests. Automated execution does not. But testing is way more than just execution. It involves interaction with other people, planning, preparations, choosing approaches, adapting to context, analysing results, acting on information, maintenance on environments, data, etc. etc. All those involve sapience, even in contexts that have heavy emphasis on automation. Time saved in automated execution is often lost in test suite upkeep, configuration management, keeping up with dependencies, etc. So human testing and automated checking are both quite time consuming after all. No real difference there. - Sami Söderblom


Automated checks validate known criteria only. Human tests go beyond known criteria into edge cases. - Dave Taddei


One is faster to check knowns and the other is slower but can explore/uncover unknowns - Luke Liu


One needs human input and the other doesn’t. That’s the main one. We can talk about what’s better in different context, gaps in one and the other and so on…but the main difference is simply that. - Patricio M.


The answer equates to creativity Vs logic. One is limitless and exhaustive while other is limited but symmetric. - Raza Sohail


Consistency (which could be good or bad); repeatability ; time ; cost ; depth ; morale ; responsibility. But mostly I would say context. - Luke Robbertse


Automation Test: Can perform with great speed compared to human but cannot perform ‘Out of the box’ until coded… - Prasoon Prakash Shukla


The critical eye - Arif Taseer Asif

3 Likes

[What’s the difference between a human test and an automated check?]

It’s like asking what’s the difference between a human and an animal and then comparing them by varying criteria, without defining the human, the animal or their contexts:

  • Speed
  • Survival instinct
  • Loyalty
  • Perseverance
  • Capacity to provide food and shelter
  • How much food and water each need
  • How much waste each creates.
    etc…

So on the question, I’d say it depends on what kind of human you’re talking about, what he knows about testing, what he tests, what’s the budget, what’s the scope, what’s the automated checking’s purpose, what resources are available, what timeframe is given, what expectations there are…

2 Likes

I am looking a bit at AI at the moment and one element I read about was that the coding for big blue which beat kasparov at chess is actually a fairly simple view of AI and that decades later what turned out to be a much harder challenge for AI was actually picking up the chess piece and moving it around the table.

For a human the latter is for most simple and the former again for most more complex, hundreds of thousands of years of evolution results in a full on sensory buffet in almost everything we do.

AI is advancing a lot and even if it could be viewed as autonomous learning, decision making and adaptability to varying degrees of uncertainty it is generally still has a very narrow field of vision, a few tasks or things to do.

An automated check is an even simpler version of this with a focused single factor assertion.

A human test on the other hand has that evolved full on sensory buffet that has a broad view of things even if it is considering the same single assertion.

Each have their strengths vs weaknesses, chess vs picking up and moving a chess piece is a simple example but perhaps the narrow vs broad comparison has some merit.

1 Like

It depends on the tester and the circumsatnces involved.
Some test houses are no better than automated tests.
Identify Test Case,
Identify entry parameters
Identify exected return.
If there are any exceptions, they are recorded as a fault.
Will a test house tester identify a spelling mistake, similarly, would an automated test set identify the same issue.
Automated tests have no memory. They are repeatable and operate at the vanilla common path level for most of us.
Testers can have a memory, and most would call on that to identify differences from one run to the next.

Common sense, heuristic and to know when to stop

Humans are really good at recognizing patterns, including when something that should be somewhere isn’t.
Humans are really good at making judgments on the fly.

Automated tests have no judgment beyond what’s programmed into them (I’m not talking AI here)
Automated tests are really good at doing the same thing the same way over and over again and not getting bored.

My short-short version is that automated tests are good things to use to speed up time-consuming, boring things like regression tests or lengthy configuration setups. Humans are good for figuring out that the UI is a really bad idea because it’s confusing or the layout doesn’t make sense.