Hi all,
This is a regular question and always be asked by the client frequently
Below are some ideas to measure automation success based on my experience:
-
We should analyze, estimate the automation difficult thing and the risk of your project. After that, you will have the number of test cases can be covered as much as possible by automation. I worked on some projects that the coverage was about 50%, and the others should be 90% or 100%. But all are successful. It’s depending on the project that you implemented automation for.
-
The framework should be clear and has full support like log/result/reports/debugs/Data-Driven. Currently, the framework should be clean code, low cost of maintenance, easy to understand and maintenance by the others and should be able to integrate with CICD tools.
-
Track the time when running the regression test by automation to make sure the automation framework decrease the time compare with the manual test. If the time is not decreased so much or be increased or the same, so we should consider doing automation or not, the framework and scripts that you implemented are not successful.
-
For the framework team, should design the framework so that it is cleaner and easier to be used by the manual QA or scripting QA.
-
The scripter should be trained the clean code and written the stable scripts. We must run automation regression test weekly or daily. So the stable scripts will help us decrease the time to invest, debug and fix the scripts.