I would like to start open discussion on what you think is the best test case management tool you have used and why?
I’ve used Testrail & Tuskr for many project recently but as QA Manager they didn’t meet my requirements as a manager nor project as whole for the following reasons. below & was forced to come up with own solutions exporting raw data into excel:
No Testcase numbers or reporting of test results per feature, only per suite (folder of many tests)
No reporting on type of testing/coverage conducted/ reporting using any custom fields
No Test case stage pipeline or filtering - Draft > sign off > Testable > automated
With regression testing (Daily BVT or smoke regression packs), there is alot of repeated tests. Report to tell you how many Test cases executed just once (how many coverage) not including repeats.
Does a testcase management tool exist that functionality to meet these requirements?
You can manage requirements and test cases, both can be organised in folders or test suites.
TestLink automatically assigns a unique intern number to test cases, which can be used for cross references and for the requirement coverage. It has great options for reporting, but I’m not sure about generating reports based on custom fields.
Test cases can have a status, like draft, or final.
You can have a separate test plan for the regression testing and then create reports from this test plan.
The reports didn’t include the fact, that some test cases were executed more than once. That was great for the overall coverage, but sometimes, we wanted an overview, how many testcase executions were done alltogether for a project, and it was really difficult to get that number.
TestLink is opensource, but it seems that it’s no longer developed and maintained.
We’ve been looking for alternatives, and among others we came across Klaros and TestMonitor, which seemed quite useful. But except from a few experiments with their free trials, I don’t have any practical experiences.
We are about to, once the system team can get it installed, trail a jira plugin by a popular vendor. And this question is worrying me a lot too @captainjonesy2 . I suspect these tools suffer from the same problem all software suffers, it solves only one perspective of the problem. Often making managers happy becomes the goal, and things like trending the pass/fail rate of a single “feature” becomes impossible unless you organize your test suites accordingly. Basically you have to have good control over how features map or how functionality/architecture maps into your test suites, you can never have both, one or the other is what I have found. I personally prefer a component mapping, but a “feature” mapping is appealing to me more and more, since that makes deprecating a feature easier to just kill off later. But does mean I will need to plan a lot better.
Being able to run one test many times in different environments is a test report vector I am keen to get working to help me find those flakey environments. So that’s a requirement for me too @monija . I was also disappointed that Testlink eventually went out of support.
I’m dreading the trial start day, because I’ll need to temper my expectations, the demos they showed us look great, but until you try things yourself, it’s really hard to know if it will help you as much as you want. Perhaps the solution is to have lower expectations. My end goal is to be able to find flakey tests quickly and easily by being able to run historical queries. So sometimes just having a decent graphing front-end is what some of us really want.
@conrad.connected - Yes you’re right, you can’t have everything. Most TC tools I’ve seen are built for audience of the testers for TC creation & execution, not for QA Managers. I’ve surprised that no one out has designed for tool for both type of users/use cases for execution & reporting.
@monija - Thanks for suggestions, very helpful. I will certainly check out TestLink, Klaros & TestMonitor.
Another requirement I have which has just occurred to me; the ability to design tests without creating dupes. My projects I’ve been on, I’ve had to create a process to ensure testers are not creating dupes leading to a waste of effect. Is a tool with a feature that would help with this?
I’m currently at looking at Xray & Zephyr which seem so far meet my earlier requirements.
@conrad.connected@monija - How do you want to decision on the right TC tool for your project? Do you have requirement checklist and see which tools meets the most of your requirements?
I’ve been prompted to write up my requirements on our internal wiki, I liked some things that Testlink did give us, but overall it was a very “expensive” maintenance-wise tool and usability was just terrible. Will update when I have a moment on what requirements goals I came up with. Busy pushing a release to the apple store today, so a tad busy.
In my old job, TestLink was already there, when I joined them. So I wasn’t participating in the decision process.
In my new job, there is no tool at all. Requirements are not explicitely documented, which means I search through dozens of tickets to find the customer’s extension requests, bugs and issues. Like a hunter and gatherer.
But I’d like to have a place where I can store and organise my harvest and haul and I need to make sure that I don’t miss anything.
So I made a list with must haves and nice to haves based on the things I knew. But in the meantime i realised, that my list was too much oriented on the workflows of my previous job. Of course in my new team everything is handled differently, so I gave myself some more time to learn and find out what will be needed here.
This was my original minimum must haves list:
Manage product requirements
Manage test cases (with steps)
Assign test cases to requirements
Organise test cases in test suites
Document test executions
Have a field for an issue-tracker-ID
Maybe later there would be more features needed:
Assign test cases to test environments
Assign test cases (or even test case versions) to product versions
Have a list of automated test cases and somehow assign them to the product requirements to make the coverage visible.
My current view on this is: maybe I don’t even need very elaborated test cases with all those test steps and expectations, but focus on a proper documentation of requirements in the first place.
Feel free to give our test management tool Testmo a try. We have built-in support for things like workflow states (e.g. Draft → Active → Retired etc.), automation linking (and filtering), coverage status columns (so you can see the latest status of all your test cases across all runs) and similar that might fit what you are looking for.
Hey Chris.
If looking at Xray & Zephyr, I hope they meet your needs. The one reason we are moving from Xray is down to their licensing (Zephyr is the same). They license on the number of Atlassian / Jira users as opposed to the number of QA / Test users. We have a team of 4 QA, yet over 300 Jira users - so the price for Xray is extortionate! If they could address this, we would ‘stick with it’ and its peculiarities.
We were looking at “Test-Gear.io” as seen at TestBash recently - but they are no longer trading!
So, “Testmo” & “Qase” are currently being considered for suitability.
We are moving away from TestRail ourselves, but I can add two tools that we have been looking at that have not been mentioned. One was PractiTest, which seemed quite robust to me, and the other was Kualitee, which is quite affordable. Unfortunately, I can’t tell you if they meet all your requirements because I don’t know them that well.
Having been a user of various tools in the past (one being TestRail, although I’ve not come across Tuskr), I’m currently using TestLodge, which works well for my current contract.
It doesn’t have as many features as TestRail, but it is very stable, and what I’ve found most helpful is the issue-tracking integrations, which work with many different bug trackers. (and will automate the entering of issue tracker ID’s)
They have the ability to create and associate requirements with Test cases; test steps can be numbered using test formatting within a test case, which are then stored in test suites. Test runs will then allow you to document and record your test executions.
I’m not sure about your additional feature list, but your must-haves are similar to what I use, so it would certainly be a contender if you are looking for a less complex tool.
We’re seeing a lot of movement toward PractiTest. To the point that we have partnered with them and also created an integration to PractiTest from our test automation tool called Alchemy Testing.
I personally have only used PractiTest a little bit, but what I have seen, I think it is great and offers many features. Certainly worth checking into.
In my last company, we used Zephyr Squad in Jira for test case management. Overall the experience was good.
User Interface was simple and for every sprint that is going on in Jira, we could parallel concurrent sprints in Zephyr and we could also add a regression suite for a particular sprint or master regression suite.
I’m not sure how much it cost to the organization but yeah it was really helpful in managing the test cases.
One of the key benefits of the Zephyr squad was that the dashboard was integrated in Jira, and once anyone navigated to the dashboard they could see all the numbers like the number of test cases executed, number of test cases not run, etc. Apart from that, we could also easily visualize the regression bugs, reopened bugs, the total number of bugs, etc. in graphs which was easy to understand as well as to share with the stakeholders.
So overall Zephyr Squad meets the requirements of our QA team and we were okay with its features and functionalities.
The first point is overcome by having the suites feature focused.
I use Azure DevOps and for the lifecycle we can add custom values in a “State” field
I like Ado with the exception of exploratory - it’s clearly design more around “scripts”.
There are some lacking in there for example history of test case results (e.g. if you wanted to use data to drive some priorities, e.g. tests that have a history of failing
I have also used TestRail in the past, but as I climbed up the ladder, I found their features to be basic.
Currently, we use BrowserStack’s test management tool in my company and it’s faring well for us. So, currently, it is my favorite. I am listing down some capabilities that really worked in our favor:
Their central, versioned repository is super easy to access across the team. Everything is much organized, so collaboration feels smooth.
Each test case has a unique ID. You can group them by custom tags and slice results by feature, sprint, or test suite.
We can define and filter by test case stages like Draft > In Review> Approved> Automated. It brings structure, especially when scaling test efforts across teams.
The next thing I like is their AI agents. They let you generate test cases from PRDs and user stories, auto-fill steps and preconditions, and convert them into low-code automated tests.
Like most other tools, it also offers smooth integrations with almost all popular automation frameworks and CI/CD tools, and of course with their suite of tools.
Now, again I cannot say if this is the right tool for you or not since, I do not know what you are your other requirements. I think BrowserStack has a trial plan, you can try it out and decide.
Wow, interested to know how does it look like for companies & individual in 2025?
Would I remain one of us , if I dare to say: I have never throughly used or feel like using any test case management tool, not that they are not useful, they absolutely are for some business, by chance I haven’t had an opportunity to use them professionally. Even though I know each one of them out there in the market.
I have been associated with large organisation & startup as well. I have had used: Asana, Redmine, Jira’s built-in feature called: “Smart Check list” & couple of others - I might be old school in that sense that I still keep all the test cases that are there in the eco system to shared Excel file. because - why not?
So, how do we manage them? How do we know whats needed to be tested and how can I guide? Answer: a defined steps for a story thats whats needed.
Stories are easy target to breakdown test cases, because it will be divided further into subtasks for specialists i-e: BE/FE/DB/Services etc.
Engaging QAs early, shift left.
Declare the test cases there and then under the story. (Spend a good time with developer 30min to 1h) I call it - QA-Kickoff.
When the work starts on that story, make sure those test cases have been covered , tested later on , engage in code reviews.
Make sure results are shared as Testing notes. either by this plug in - or in simply text based jira field called: Testing Notes. Simple and easy.
And time to time I do visit the test cases to update/edit as they are part of entire test strategy not just one feature and everyone refers to them when needed but in reality the test cases written under stories are suffice and targeted.
Its the ‘Quality process model’ I had proposed and we follow. I believe it must be different with different team structure, thats the beauty of Agile, choose what works best for you!
, if I dare to say: I have never throughly used or feel like using any test case management tool, not that they are not useful, they absolutely are for some business, by chance I haven’t had an opportunity to use them professionally. Even though I know each one of them out there in the market.
because - why not?