What does the term Automation mean to you?

(Heather) #1

When I started in testing, the only thing I thought automation meant was Selenium. I’ve since learned that automation can mean so many things! Unit tests are automation, API checks, even JMeter scripts. Selenium isn’t the only thing under the umbrella of automation.

I’m interested to hear from others in the community to see what does the term Automation mean to you?

What does the future hold for a manual tester moving towards automation?
(JC) #2

The main thing I notice is that most people outside QA don’t quite get the whole creation/maintenance/execution resource overhead, especially when the system you’re working on is changing all the time.

(Chris) #3

Automation is manual, exploratory testing helped by a tool that performs coded checks.

(Steve) #4

Automation testing is a misnomer really. Automated tests need to be manually thought of, manually written, and then they can be automatically scheduled to execute, so the only ‘automated’ part is the test execution. So to me, Automation means ‘the automated execution of test scenarios’. If in the future, we are able to automatically generate tests, and have them run themselves, then the term would have a broader meaning.

(Garry) #5

Automation in terms of software development can also apply to automatic build creation and deployment. However, in term of QA we usually automate tests which are needed to be executed more than once. There is no point in automating a test which needs to be executed only once! Automation is initially attributed to high cost but its real benefit is realised in regression testing.

(Darrell) #6

To me automation means the creation of scripts which allow for repetitive actions to be run again and a again.

If I want to ensure business features continue to work as new features are introduced then I might write Selenium automation to test things and use a gherkin based tool to express the business features (Cucumber, JBehave, etc.). As new features are created, my UI Test Automation will ensure older features continue to work.

I can also use automation to run unit tests. Run static analysis tools on the code. There is also automated code coverage, performance testing, code complexity analysis, etc.

We can create automation to prompt the developer to add comments (provide them with a helpful template) when checking in code and even require them to include a ticket number to cross reference it to the story or defect report.

Setting up of the data before we start testing can be automated. SQL scripts to load a database. This can be from scratch or we could take production data, modify it to hide sensitive information then load it into a test environment.

If we are deploying the software to a cluster of machines, I can use automation to ensure all the machines were deployed correctly. I write a script to check 1 machine then make it loop to check 300 machines.

Current client has APIs to query product, province, language, etc. If I want to check all possible combinations this is definitely a candidate for automation. If it is Soap or REST calls I could use a tool like SoupUI to test everything.

Any time I feel I’m testing the same thing (at any stage of development) multiple times or on current body with multiple parameters then I think about how can I automate that.

A shell script to check for magic numbers or magic strings is a quick automation I can use. If I know how to use Perl or Python, I might whip together a quick Per hack to see of there is an issue with the current code base (e.g. we should never use the number 0002000544 as this is a deprecated account; find src/ -name “*” -type f -exec grep -H “0002000544” {} ‘;’ ).

The list of things I automate is endless. Some is just short lived to test a theory or check some data. Other is something I’ll run over and over for each check in or each release.

(Cassandra) #7

Patrick Prill helped me realise that, when I see / hear “automation”, I have a habit of changing that to “automation in testing” in my mind :slight_smile:

But automation doesn’t necessarily need to have anything to do with testing, even if it aids testers. Automation is something that’s automatic, anything. I think sometimes we’re a little too focussed on our assumptions, or the stand-points we’re so used to defending / explaining and this can restrict us from looking at other angles.

(Jesper) #8

Perhaps there is a difference if there is an assert() or not.

If I have a SOA tool that I can use to look up data it’s not automation - just plain ol’ testing.
If I have soce code that asserts(stuff) - then it’s test automation…

(Gustavo) #9

I come from an Industrial Engineering background so I think I have a more broad understanding of concept of what automation is.

Automation to me is to rely REPETITIVE TEDIOUS tasks that used to be performed by humans to some kind of technology. In the manufacturing industry that would be robots for example.

But as someone said before, it still needs human input. In the manufacturing arena, there’s technicians who program the robots, who service them, who mantains them, who fix problems. In software testing, we need to check results and try to find a meaning, detect trends, bugs, etc. We need to develop and mantain them, etc.

Automation is only another tool for testing, a very powerful one, but still just a tool. Because if the automated tests run and no tester is there to interpret the results, alert, escalate, prioritize bugs detected well those issues are not going to be fixed

(Marllos) #10

I really enjoy your questions! Very simple and insightful! Thanks.

(Stefan) #11

In my mind Automation is anything that is done with the help of features of some tools, which otherwise would have to be done ‘by hand’ with a mouse, keyboard and lots of precision/attention/patience when doing it:

  • filtering entries in an excel file,
  • using a key-shortcut through a library in SublimeText that formats a one line xml from a log file into a well formatted version easily readable,
  • filtering network traffic through Wireshark based on a range of ip’s,
  • running a suite of scripts for environment setup,
  • doing an sql query,
  • scheduling jobs to run a script at certain times,
  • using a tool to monitor the device resources while running an endurance script…

The opposite of automation:

  • scrolling through a database to find an entry instead of doing a query;
  • monitor the device resources while running an endurance script - by noting down in a text file every x minutes the observed hdd/cpu/memory values from task manager, while in between stressing the application with a mouse/keyboard.
  • instead of scheduling jobs to run a script at certain times - do the task by hand at the desired times.

And related to testing - to me automated testing doesn’t exist. The closest it gets is as kinofrost mentioned above: doing testing with the help of some tools.