Test case management tool - JIRA Plug-in

Hi community, first time posting so please be kind.

Just a bit of background, I came from traditional orgs with waterfall methodologies following agile principles. Now I have moved into a scale up / start up with DevOps / Lean methodologies with 2-week sprint releases.
To speed up our automation, we have re-factored our automation tests from Selenium to Cypress and oh boy, it was significantly faster and quicker to create / troubleshoot test scripts.
Now comes the test case management tool, which at this stage, is more of a nice-to-have since our test cases are basically our Cypress spec files and our test result reporting is TeamCity, which is our CI/CD tool. We map our test scenarios via JIRA stories as acceptance criteria.
Our management has given us a go-ahead to PoC some tools, with obvious justifications before we procure the decided tool. I have used Zephyr (and ZAPI) to automate our Selenium test scripts and results generated in JIRA in my previous stint (It even creates a unique JIRA Bug when a run fails). I’ve heard good reviews on XRAY as well. We basically shortlisted between these two tools (We decided to make it a JIRA plug-in for easier automation and more centralised).
Can people share experience on both these JIRA plug-ins and what makes the other different?
Also, the reason looking for a test case mgmt tool at this stage is because of the following:

  • No need to collaborate as we only have 2 QA’s against 8 devs so we can just chat / discuss what the test scenarios are or who is working on which test script to automate.
  • Only two test environments
  • Reporting or failed scripts are already recorded in TeamCity builds.

However, we want to be prepared as there is a high chance of the team growing and usually, when the team scales, it gets a bit too late and these things get buried in the tech debts.

Anyone has time to share their experience?


I’m in a similar new work process, just minus the automation. I’m creating test cases to give the product its first ever proper, ground-up test (I’m their first tester), and part of that meant choosing a test case management tool. We don’t use JIRA here (Azure DevOps instead) but I did work with JIRA at my last place. There, they used TestRail to house test cases, with an integration to JIRA that was loosely used to link tickets, and it seemed very useful. They also used it as a reporting tool for the automation scripts run by the automation testers. I’ve never used Zephyr or XRAY but I’ve heard good things.

In my current place, I considered (and tried) qase.io, TestPad, CucumberStudio, and TestRail for the test case management tool. They seemed to have baseline similarities, with different features for different use-cases. CucumberStudio looked like it could have good automation integration, but I ended up choosing TestRail because it has the most relevant features for what my needs are, and caters for expansion and re-usability. I did a quick google and it seems it can be used with Cypress, so that could be something to consider.

I’m sorry if my reply hasn’t helped at all, but there could be a kernel of something useful in there?

I’m currently using Jira and we have integrated it with TestRail and found it very useful.

Test Rail was highly recommended by an ex-colleague who had used it in a previous role. It’s easy to work with and not overly complicated.

Test Rail’s API is pretty comprehensive, so you can do pretty much anything via the API.

We have been using it for over a year now and its always improving which is nice to see. We use it for both our human testing and the automated tests using Gherkin syntax.

We use the API to create new test cases in TR and also to add the results from automated tests.

Hope you find what works for you.

Thanks both. Seems like TestRail is a popular tool. Our company actually has TestRail but I found it similar to qTest where it is platform on its own. You need separate credentials with JIRA. This creates a bit of an issue with different stakeholders (Product owners, Marketing, Design, etc.) needing to see test results.

Also, I checked that our TestRail was last used was a year ago so it seems the company is paying for something not being used and has low adoption rate from QA’s.

I like synapseRT plugin. It is cheaper than Zephyr and was better at the time of evaluation. Using its REST API, you can update Selenium script results in Jira. You do not need to learn another GUI, like TestRail. You stay within Jira since Test Case and Test Plan just two new Jira issue types.