Tools to automatically record test metrics


(Becky) #1

Hi guys,

I am a new automation tester at a small firm using the following tools

Robot Framework
Pycharm
Jira
Confluence
There is also talk of ingratiating Jenkins

Currently after every run of our test suit (triggered using a .bat file) we manually view the results file generated by Robot framework and update the following information into a spreadsheet.

Number of tests run
Number of tests Passed/Failed
A note as to why any tests may have failed.
Why the tests where run i.e. Smoke/daily run
A link to the local file that stores the report.html

This cannot be a good way of recording this information.

My question is this.

What tools do you use to store test result metrics?
is there any way to automate this process, ideally with open source software or Microsoft office?
I have thought about creating some sort of .bat file that grabs the results data and updates the spreadsheet, has anyone done this?

Thanks guys


(D) #2

If you are going to integrate Jenkins this may be of use (I’m not too familiar with Robot Framework/Jenkins so sorry if going down the wrong path!):

https://wiki.jenkins.io/display/JENKINS/Robot+Framework+Plugin

Also seems there is this app in the Atlasian Marketplace, not sure if any use for your needs?


(Burdette) #3

In a recent gig, I used Ruby to read the log files and update a History Report (Excel spreadsheet).

In previous gigs, I have done the same in Python (also downloading log files from Jenkins).

In Ruby, I used REXML to read the log files and WIN32OLE to update the report.

At the left of the report are the verdict IDs, log file paths, etc.; a the top are the dates, build IDs, counts, etc.

There’s a column for each test run, giving the verdict outcome (passed/failed/blocked).

The update inserts a new column for the new test run, pushing the previous data to the right, so the history for each verdict reads left-to-right present-to-past.

In my GitHub project, RubyTest, I have already built code to generate the Changes Report, but have not yet built code to build/update a History Report. Should that be a priority?


(Shiva) #4

Robotframework-metrics report has export functionality where we can export suite, test, keyword metrics in different formats (pdf, excel,csv).

Sample Report