How do you measure automation success?

🗄️ Archive
23

Automation is successful if you are able to deliver often at a sustainable pace.
Automation is not successful if you are able to deliver often at an unsustainable pace
Automation is not successful if you are not able to deliver often

HTH

1 Like
25

As Richard Bradshaw proposed easy way to measure is using the Mnemonic TRIMS
T- How Targeted your tests are [What are they covering Business Risks, Core Modules or Test cases] ?
R- Reliable How reliable your Automation Suite is, Results are ? Are these results accurate when reproduced in production
I-Informative How informative your tests and results are: Can Functional testers see what a particular Automated test is testing, Can business users understand/read the Test Results to assess what’s failed and what it means to the business? No one really cares about passed one’s and Green ticks :grinning:
M- Maintenance :smile: How easy is to maintain your Automation Suite for any Test Engineer within the team or cross skilled Developer who is willing to help? How many times scripts failed due to locator changes or other updates, Avg. Effort spent in last month to fix/refactor/update the Automation Suite.
S- Speedy: How Speedy and optimized your tests are: Total Test execution time, how this change the Software Test delivery for your organization, metrics around time to production before and after. How many parallel browsers can they be run on at a given time? Anything that can be improved here like utilization of resources, infrastructure costs etc.

Overall, the main success from Automation for an organization comes two key things from my view

  1. Automate Appropriately- It’s not a race or silver bullet for all problems please empower teams that everything is not suitable or right to Automate. Identifying what’s right to automate and what’s not is key.
  2. Regression should be Reliable- Maintain your Automate suite like a Master code branch ready to execute any time and reliable to produce release readiness metrics.
26

Hello,

Very useful question.

Now in day’s as an Automation Tester, it is important to measure the success of the plan otherwise you will just work every day and but not smarter.

While work on Test Automation There are various points I would like to suggest you to consider to measure your success.

There are the 3 most important Metrics you should consider

  • Activity Metrics
  • Efficiency Metrics
  • Value Metrics

I thought thses are the key performance indicators that will help you to gain value while working on Automation Testing