30 Days of Automation in Testing Day 1: Definitions

(Heather) #1

Today’s challenge in 30 days of automation in testing is:

Look up some definitions for ʻAutomationʼ, compare them against definitions for ʻTest Automationʼ.

Celine on the Ministry of Testing Slack channel was in early with this one :grin:

"https://www.qasymphony.com/blog/test-automation-automated-testing/#

Automated testing is the act of conducting specific tests via automation (e.g. a set of regression tests) as opposed to conducting them manually, while test automation refers to automating the process of tracking and managing the different tests.

Both automated testing and test automation are important to continuous testing, but it’s the latter that really takes the cake on this one."

Over to you, what definitions have you found?

(Ramakrishna) #2

Automation (Google > define: Automation) : the use or introduction of automatic equipment in a manufacturing or other process or facility.

Automation (Wikipedia) : Automation is the technology by which a process or procedure is performed without human assistance.Automation or automatic control is the use of various control systems for operating equipment such as machinery, processes in factories, boilers and heat treating ovens, switching on telephone networks, steering and stabilization of ships, aircraft and other applications and vehicles with minimal or reduced human intervention. Some processes have been completely automated.Automation covers applications ranging from a household thermostat controlling a boiler, to a large industrial control system with tens of thousands of input measurements and output control signals. In control complexity it can range from simple on-off control to multi-variable high level algorithms.

Test Automation: (Wikipedia) : In software testing, test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.There are many approaches to test automation, however below are the general approaches used widely:

Graphical user interface testing: A testing framework that generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct.

API driven testing: A testing framework that uses a programming interface to the application to validate the behaviour under test. Typically API driven testing bypasses application user interface altogether. It can also be testing public (usually) interfaces to classes, modules or libraries are tested with a variety of input arguments to validate that the results that are returned are correct.

Test automation tools can be expensive, and are usually employed in combination with manual testing. Test automation can be made cost-effective in the long term, especially when used repeatedly in regression testing. A good candidate for test automation is a test case for common flow of an application, as it is required to be executed (regression testing) every time an enhancement is made in the application. Test automation reduces the effort associated with manual testing. Manual effort is needed to develop and maintain automated checks, as well as reviewing test results.

(AMIT) #3

Definition of Automation. (Source: https://www.techopedia.com/definition/32099/automation)
Automation is the creation of technology and its application in order to control and monitor the production and delivery of various goods and services. It performs tasks that were previously performed by humans. Automation is being used in a number of areas such as manufacturing, transport, utilities, defense, facilities, operations and lately, information technology.

Definition of Test-Automation. (Source: https://www.techopedia.com/definition/17789/test-automation)
Test automation is a term used in software testing and some other kinds of IT related testing or quality assurance. It means that there is no human input required to generate a test. Different kinds of test automation help businesses to pursue goals like software testing with fewer resources, or in more efficient ways.
At its core, test automation means that no human has to manually initiate the test. Even many non-technical users will be familiar with this principle through looking at tools like anti-virus programs, which self-initiate periodic testing of a computer or workstation. Virus scans and other events also happen automatically. In the same way, test automation allows software testing processes to happen automatically, without the burden of user initiation.

Beyond test automation, specific kinds of innovation help to make software testing even easier and more efficient. For example, some companies are talking about scriptless test automation. This involves a system where programmers do not need to write a script for each specific individual test case. By using reusable code assets, scriptless test automation resources can also help to provide less labor-intensive ways for companies to test software or otherwise evaluate products and services.

(Peggy) #4

Here is what I wrote:

I’m participating in the 30 Days of Testing Automation challenge. Today is the first day.

The challenge today is to look up the definitions of “automation” and compare them against definitions for “test automation”.

Dictionary.com defines automation as:

the technique, method, or system of operating or controlling a process by highly automatic means, as by electronic devices, reducing human intervention to a minimum.
a mechanical device, operated electronically, that functions automatically, without continuous input from an operator.
act or process of automating.
Wikipedia has this to say about test automation; "

“In software testing, test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.”

If you compare the definition of automation and test automation, they both speak to the idea of automating repetitive tasks so that human interaction with them is at a minimum. Automation requires the majority of the human interaction in the planning and execution phase ideally. You need to have a tester(s) decide what tests they are going to automate and decide what tools will be the best to use. The tester then needs to write the tests, decide how often they are going to run and how to monitor the test results. You also have to take into account maintenance of the tests, such as if the code changes and you need to adjust your tests. Over time this definitely has the potential to reduce human interaction but I am of the opinion testing is always going to need human interaction. Automation is just one tool that a tester needs to have in their toolbox. Mastery of automation allows you to free up your time as a tester to go into depth on new features and to test user workflows and really delve into how your customer is using the product and design workflows that will let you test accordingly. That is where you really start to add value as a tester.

(Kumar) #5
(Lulu) #7

Definition of Automation ( Source:http://www.dictionary.com/browse/automation)
The technique, method, or system of operating or controlling a process by highly automatic means, as by electronic devices, reducing human intervention to a minimum.

Definition of Test automation (Paul Ammann and Jeff Offutt)
The use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions

(Magda) #8

For a general definition of automation, I figured I would reach for the fabled German engineering thought for input. Admittedly, I started with Wikipedia, but it did not disappoint. There is no automation like industrial automation!

Automation
According to that definition, automation in general is both the process and its result, where repeatable actions or workflows with a specific purpose are executed entirely or mostly without human intervention.
Importantly, there are three main areas of automation:

  • monitoring/measuring,
  • process control (open loop, where an action is repeated a set number of times or an input state held maintained for a specified time),
  • feedback control (where a set output state is maintained or hovered around by reading the state, adjusting inputs and then checking for the resulting output state).
    A key feature of such automation is the two-way flow of information (from sensors and to controlling elements) and adaptive behavior.

Test automation
Meanwhile, test automation is in some sources the use of automated tools to support any testing-related activities.
Generating test data and driving the software under test are obvious candidates, but I was surprised to also learn that for example test case management as well as reporting test results are considered to be such use. I guess it makes sense in a way but to me it feels more like mechanisation rather than automation, where tools make the “physical” work easier but the flow and actions are still executed by a human.

However, the crucial difference between the two seems to be the feedback loop which is not a part of test automation itself. Of course the test developer can and should take the observed results into account and dig deeper around the interesting ones - that’s, after all, how automated tests support exploration - but the advanced industrial automation systems are themselves designed to take the observed result and adjust inputs so that it better matches the set target state.
That state is of course numerical, a specified payload temperature or pressure in a tank, and in production environment definitely not to be experimented with. With the growing interest in using AI for testing software though, one can imagine setting a neural network the goal of finding behaviours which deviate from a specification, be it to gather knowledge about the application under test or just plain to find bugs.
For now, paradoxically, it still requires a human to reach the levels of what true automation can do elsewhere already :woman_shrugging:

(David) #9

Since I’m late and feel like I have to be different somehow :stuck_out_tongue: I looked for some sort of technical dictionary. A bunch of stuff from https://whatis.techtarget.com came up for different types of “automation”

Labor automation is the practice of substituting technology for human labor to perform specific tasks or jobs. Automation involves mechanization but also expands it by using technology to further remove people from tasks. The term today generally applies to cases in which the technology being deployed replaces knowledge-based tasks or positions, rather than merely replacing manual processes.

Network automation is a methodology in which software automatically configures, provisions, manages and tests network devices. It is used by enterprises and service providers to improve efficiency and reduce human error and operating expenses.

Industrial automation is the control of machinery and processes used in various industries by autonomous systems through the use of technologies like robotics and computer software.

IT automation is the use of instructions to create a repeated process that replaces an IT professional’s manual work in data centers and cloud deployments. Software tools, frameworks and appliances conduct the tasks with minimum administrator intervention. The scope of IT automation ranges from single actions to discrete sequences and, ultimately, to an autonomous IT deployment that takes actions based on user behavior and other event triggers.

Then of course we have:

Automated software testing is a process in which software tools execute pre-scripted tests on a software application before it is released into production.

Short and to the point, I guess?

Also here’s a fun twist. Here’s the first image that comes up in a search for “automation”

image
image.jpg696×464 98.3 KB

Here’s the first that comes up for “test automation.”

image

-Dave K

(Heather) #10

From our friends participating on Twitter

(Paul) #11

Please allow me to offer a polite and respectful critical perspective by stating my opinion that the phrase “test automation” (and synonyms such as “automated testing”) is flawed and contentious. It is inevitably vague (it has been applied to any testing at all where some tool has been used at least once) and I believe only works by provoking a tendentious difference and value judgement against “manual” test methods - which has poisoned many conversations about the application of automation methods.

I wrote a blog article in February on dialectics in testing which puts forward an idea based on Derrida to explain why terms like this can be so troubling.

(Priyanka ) #12

Definition of Automation(source: https://www.isa.org/about-isa/what-is-automation/)
Automation
According to that definition, Automation as "the technique of making an apparatus, a process, or a system operate automatically.”

We define automation as "the creation and application of technology to monitor and control the production and delivery of products and services.”

Definition of Automation testing(source: http://www.softwaretestingmentor.com/introduction-to-automation/)
Automation testing
According to that definition, Using Automation tools to write and execute test cases is known as automation testing. No manual intervention is required while executing an automated test suite.

Testers write test scripts and test cases using the automation tool and then group into test suites.

(João Farias) #13

Starting out late this month, but here is some quick thoughts on it:

(Karishma) #14

#Day 1:Definations

Automation is to make a process in a factory or office operate by machines or computers, in order to reduce the amount of work done by humans and the time taken to do the work.

Test Automation uses special software tools to automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.

Test Automation Benefits

  1. Faster Feedback
    Automated testing comes as a relief for validation during various phases of a software project. This improves communication among coders, designers, and Product Owners, and allows potential glitches to be immediately rectified. Automated testing assures higher efficiency of the development team.

  2. Accelerated Results
    Owing to the quick implementation of automated testing, plenty of time is saved even for intricate and enormous systems. This allows for the testing to be carried out repeatedly, delivering faster results each time with lesser effort and time.

  3. Reduced Business Expenses
    It comes as no surprise that while the initial investment may be on the higher side, automated testing saves companies many a penny. This is predominantly due to the sharp drop in the amount of time required to run tests. It contributes to a higher quality of work, thereby decreasing the necessity for fixing glitches after release and reduces project costs.

  4. Testing Efficiency Improvement
    Testing takes up a significant portion of the overall application development lifecycle. This goes to show that even the slightest improvement of the overall efficiency can make an enormous difference to the overall timeframe of the project. Although the setup time takes longer initially, automated tests eventually take up significantly lesser amounts of time. They can be run virtually unattended, leaving the results to be monitored towards the end of the process.

  5. Higher Overall Test Coverage
    Through the implementation of automated tests, more tests can be executed pertaining to an application. This leads to a higher coverage that in a manual testing approach, would imply a massive team limited heavily with their amount of time. An increased test coverage leads to testing more features and higher quality applications.

  6. Reusability of Automated Tests
    Due to the repetitive nature of test automation test cases, in addition to the relatively easy configuration of their setup, software developers have the opportunity to assess program reaction. Automated test cases are reusable and can hence be utilized through different approaches.

  7. Earlier Detection of Defects
    The documentation of software defects becomes considerably easier for the testing teams. This helps increase the overall development speed while ensuring correct functionality across areas. The earlier a defect is identified, the more cost-effective it is to fix the glitch.

  8. Thoroughness in Testing
    Testers tend to have different testing approaches, and their focus areas could vary due to their exposure and expertise. With the inclusion of automation, there is a guaranteed focus on all areas of testing, thereby assuring best possible quality.

  9. Faster Time-to-Market
    Test Automation greatly helps reduce the time-to-market of an application by allowing constant execution of test cases. Once automated, the test library execution is faster and runs longer than manual testing.

  10. Information Security
    The effectiveness of testing will be largely dependent on the quality of the test data you use. Manually creating quality test data takes time, and as a result, testing is often performed on copies of live databases. Automation solutions can help with creating, manipulating, and protecting your test database, allowing you to re-use your data time and again. The time and cost savings in this area are potentially huge.

The most promising aspect of the automated testing process is that it adds value to all of the stakeholders involved. It improves brand image, generates higher revenue, increases brand recall and ensures higher customer retention. As a result, there is an increase in the investment on product research and process innovation, thereby helping the organization to scale new heights and establish superiority in the market.

(Evrim) #15

Automated test refers to a test that conducts in automation.
Test automation refers to all test process that manages in automation.
So test automation covers automated tests.

(rosiecorn) #16

Starting out late this month, but here is some quick thoughts on it:

Automation

Automation is the technology by which a process or procedure is performed without human assistance

Manual Test

Be done in person, by clicking through the application or interacting with the software and APIs with the appropriate tooling
It is very expensive as it requires someone to set up an environment and execute the tests themselves
It can be prone to human error as the tester might make typos or omit steps in the test script

Automation Test

Using an automation tool to execute your test case suite
Be performed by a machine that executes a test script that has been written in advance.

Tests can vary a lot in complexity, from checking a single method in a class to making sure that performing a sequence of complex actions in the UI leads to the same results.

It is a key component of continuous integration and continuous delivery
it’s a great way to scale your QA process as you add new features to your application.

It’s much more robust and reliable than automated tests – but the quality of your automated tests depends on how well your test scripts have been written.

But there’s still value in doing some manual testing with what is called exploratory testing

The benefits of automation testing

• Faster than the manual testing
• Wider test coverage of application features
• Reliable in results
• Ensure Consistency
• Saves Time and Cost
• Improves accuracy
• Human Intervention is not required while execution
• Increases Efficiency
• Better speed in executing tests
• Re-usable test scripts
• Test Frequently and thoroughly
• More cycle of execution can be achieved through automation
• Early time to market

(Helen) #17

Based on https://www.techopedia.com/definition/32099/automation : Automation is the creation of technology and its application in order to control and monitor the production and delivery of various goods and services. It performs tasks that were previously performed by humans. Automation is being used in a number of areas such as manufacturing, transport, utilities, defense, facilities, operations and lately, information technology.
Business intelligence is an example of application of automation in software technology

And Test automation is defined as follows:
Automated testing or test automation is a method in software testing that makes use of special software tools to control the execution of tests and then compares actual test results with predicted or expected results. All of this is done automatically with little or no intervention from the test engineer. Automation is used to to add additional testing that may be too difficult to perform manually

So the comparison seems simple , Automation has a broad category different sectors apply automation like Industry, Agriculture, Sports, Information technology ,etc uses automation , where test automation is a subset of Automation which is applicable to Information technology area specific to Software testing.

(Pablo) #18

Fast-track answer:
Automation - not to repeat the answers given, but to paraphrase:

the efficient means by which an otherwise manual task is replaced by a tool or machine for the purposes of completing consistent work in an efficient manner.

Test Automation - can then be described as something that is:

  • programmed - in a language that can be human-readable (like Behavior-driven Tests) or code-laden
  • reusable in such a way that the variables and test data can be mutable, but the script remains minimally changed
  • reliable and resilient to code-changes; easily maintained as the project evolves
  • repeatable so that the tests deliver the same results in the same way locally or as part of an integration program
  • cost-effective in terms of both development and deployment
(Quang Le) #19

Automation:
I believe, the term – “Automation” generically means just automating a process that is otherwise carried out manually with the help of human effort. Automation could be achieved by the introduction of machines/ scripts/ bots/ robots to a process with an intention to minimize the manual human effort resulting in saving time and cost.
Automation is the creation of technology and its application in order to control and monitor the production and delivery of various goods and services. It performs tasks that were previously performed by humans. Automation is being used in a number of areas such as manufacturing, transport, utilities, defense, facilities, operations and lately, information technology.

Test Automation:
Test Automation is often misconceived as just automating a test case. But I believe this term – “Test Automation” has a bigger purpose and meaning. Test Automation means automating the process of testing to achieve continuous testing with the motive to assure the quality of a software continuously in an autonomous fashion with minimal manual intervention.
For example, in the information technology domain, a software script can test a software product and produce a report. There are also various software tools available in the market which can generate code for an application. The users only need to configure the tool and define the process. In other industries, automation is greatly improving productivity, saving time and cutting costs.
Automation is evolving quickly and business intelligence in applications is a new form of high-quality automation. In the technology domain, the impact of automation is increasing rapidly, both in the software/hardware and machine layer. However, despite advances in automation, some manual intervention is always advised, even if the tool can perform most of the tasks.

(Mark Winteringham) #20

A lot of the sources that I might go to for ‘Automation’ have already been listed in this thread, so I am going to be a bit cheeky and move straight onto my personal definition of ‘Test Automation’

AUTOMATION IN TESTING
            is a means
TO AUGMENT MY TESTING

I use this definition because it puts me in the centre of whatever automation activities are being carried out. They work to serve some goal I am personally trying to achieve. They are not, however, attempting to replace me.

This approach also allows me to be critical of my automation through the use of models such as McLuhan’s Laws of Media (Big thanks to Michael Bolton for putting me onto this). McLuhan claims that everything that extends our human abilities such as Power tools, TV, Language, the Internet, Cars, etc. is a media and that means they are subject to the Laws of Media. Automation in testing is no different, I use automation to extend my testing abilities, so therefore they are a Media as well.

That way I can observe how they EXTEND good and bad testing behaviour, a lot of automation is poorly designed because there are incorrect assumptions made about test design. If I overuse a tool how can it’s usefulness REVERSE and mean I am swamped with maintenance, have to sift through mountains of data, lose work and time due to bugs in the tool I am using. What tools does it OBSOLESCE, whether that’s another automated tool OR a mental tool I use? Finally, I can think of what other types of testing my automation reminds me of, which might help me RETRIEVE ideas such as a data-driven testing tool is also doing stress testing at the same time, is that a good idea that it’s doing both?

Reference: https://www.provenmodels.com/18/four-laws-of-media/marshall-mcluhan/