What does the term Automation mean to you?

When I started in testing, the only thing I thought automation meant was Selenium. Iā€™ve since learned that automation can mean so many things! Unit tests are automation, API checks, even JMeter scripts. Selenium isnā€™t the only thing under the umbrella of automation.

Iā€™m interested to hear from others in the community to see what does the term Automation mean to you?

3 Likes

The main thing I notice is that most people outside QA donā€™t quite get the whole creation/maintenance/execution resource overhead, especially when the system youā€™re working on is changing all the time.

2 Likes

Automation is manual, exploratory testing helped by a tool that performs coded checks.

1 Like

Automation testing is a misnomer really. Automated tests need to be manually thought of, manually written, and then they can be automatically scheduled to execute, so the only ā€˜automatedā€™ part is the test execution. So to me, Automation means ā€˜the automated execution of test scenariosā€™. If in the future, we are able to automatically generate tests, and have them run themselves, then the term would have a broader meaning.
Steve

2 Likes

To me automation means the creation of scripts which allow for repetitive actions to be run again and a again.

If I want to ensure business features continue to work as new features are introduced then I might write Selenium automation to test things and use a gherkin based tool to express the business features (Cucumber, JBehave, etc.). As new features are created, my UI Test Automation will ensure older features continue to work.

I can also use automation to run unit tests. Run static analysis tools on the code. There is also automated code coverage, performance testing, code complexity analysis, etc.

We can create automation to prompt the developer to add comments (provide them with a helpful template) when checking in code and even require them to include a ticket number to cross reference it to the story or defect report.

Setting up of the data before we start testing can be automated. SQL scripts to load a database. This can be from scratch or we could take production data, modify it to hide sensitive information then load it into a test environment.

If we are deploying the software to a cluster of machines, I can use automation to ensure all the machines were deployed correctly. I write a script to check 1 machine then make it loop to check 300 machines.

Current client has APIs to query product, province, language, etc. If I want to check all possible combinations this is definitely a candidate for automation. If it is Soap or REST calls I could use a tool like SoupUI to test everything.

Any time I feel Iā€™m testing the same thing (at any stage of development) multiple times or on current body with multiple parameters then I think about how can I automate that.

A shell script to check for magic numbers or magic strings is a quick automation I can use. If I know how to use Perl or Python, I might whip together a quick Per hack to see of there is an issue with the current code base (e.g. we should never use the number 0002000544 as this is a deprecated account; find src/ -name ā€œ*ā€ -type f -exec grep -H ā€œ0002000544ā€ {} ā€˜;ā€™ ).

The list of things I automate is endless. Some is just short lived to test a theory or check some data. Other is something Iā€™ll run over and over for each check in or each release.

1 Like

Patrick Prill helped me realise that, when I see / hear ā€œautomationā€, I have a habit of changing that to ā€œautomation in testingā€ in my mind :slight_smile:

But automation doesnā€™t necessarily need to have anything to do with testing, even if it aids testers. Automation is something thatā€™s automatic, anything. I think sometimes weā€™re a little too focussed on our assumptions, or the stand-points weā€™re so used to defending / explaining and this can restrict us from looking at other angles.

6 Likes

Perhaps there is a difference if there is an assert() or not.

If I have a SOA tool that I can use to look up data itā€™s not automation - just plain olā€™ testing.
If I have soce code that asserts(stuff) - then itā€™s test automationā€¦

I come from an Industrial Engineering background so I think I have a more broad understanding of concept of what automation is.

Automation to me is to rely REPETITIVE TEDIOUS tasks that used to be performed by humans to some kind of technology. In the manufacturing industry that would be robots for example.

But as someone said before, it still needs human input. In the manufacturing arena, thereā€™s technicians who program the robots, who service them, who mantains them, who fix problems. In software testing, we need to check results and try to find a meaning, detect trends, bugs, etc. We need to develop and mantain them, etc.

Automation is only another tool for testing, a very powerful one, but still just a tool. Because if the automated tests run and no tester is there to interpret the results, alert, escalate, prioritize bugs detected well those issues are not going to be fixed

2 Likes

I really enjoy your questions! Very simple and insightful! Thanks.

In my mind Automation is anything that is done with the help of features of some tools, which otherwise would have to be done ā€˜by handā€™ with a mouse, keyboard and lots of precision/attention/patience when doing it:

  • filtering entries in an excel file,
  • using a key-shortcut through a library in SublimeText that formats a one line xml from a log file into a well formatted version easily readable,
  • filtering network traffic through Wireshark based on a range of ipā€™s,
  • running a suite of scripts for environment setup,
  • doing an sql query,
  • scheduling jobs to run a script at certain times,
  • using a tool to monitor the device resources while running an endurance scriptā€¦

The opposite of automation:

  • scrolling through a database to find an entry instead of doing a query;
  • monitor the device resources while running an endurance script - by noting down in a text file every x minutes the observed hdd/cpu/memory values from task manager, while in between stressing the application with a mouse/keyboard.
  • instead of scheduling jobs to run a script at certain times - do the task by hand at the desired times.

And related to testing - to me automated testing doesnā€™t exist. The closest it gets is as kinofrost mentioned above: doing testing with the help of some tools.

Hi

Whether it is Web UI, API or salesforce lwc testing etc, the basic goals of the automation is

  1. To minimize human effort by automating running tests.
  2. Reducing the cost of testing without putting quality at stake.

Selenium is just a tool provided to mainly automate the UI of the application. But in actual everything else from front-end to back-end can be automated for testing.

For me reporting of test cases after automation is the second most important thing.
Passed test cases surely gives you the idea of a stable system, but the report of failed test cases help you to point out any bug/error.

So for me automation is running my test cases continuously for me to make sure my system is stable, but if my system has a bug then I should get a detailed report regarding itā€™s how and where.

The picture is still work-in-progress, because I still see some ambiguity.

In general I observe different meanings.
I see it problematic to lable nearly any code / script as automation. Then it becomes identical to development.
I use the phrase automation only in relation to user actions (via UI or API).