I’m searching for a testing management tool which is listed as “the best one”. I need your advices and your thoughts about the tool you are using. And would you like to answer this question: " Which features of a test management tool that make you choose/use them?" Thank you guys!
1 Like
There is no “best one”, but there is probably a best fit for your situation.
Some projects I do involve a range of business users doing testing, here some tools shine with features like mobile versions and automatic screen capture.
Some projects I do involve large code bases in Visual Studio and agile interactions with the developers. Here linking work items together is important.
there is TestRail, VSTS, Panaya, Plutora, Tricentis Tosca and many more…
2 Likes
Thanks for your useful info. And can I ask you a question? Which feature of a test management tool that make use it?
sure 
I prefer tools that have a requirements module (or integrations to the same), a hierarchy of test activities, tests (scripts, cases… YMMV) and bugs (or a reference to bug tracker). Most of the above tools have this.
But what make me choose the final tool is a matter of project context.
2 Likes
We have been using Kualitee for our test & bug management needs for quite some time now. The tool supports both manual and Automation testing. It has integration with Selenium for test automation and Jenkins & Bitbucket pipelines for release management.
The tool is also free for a small team of 3 people or add up to 10 user for $45. You can create tester, developer and QA manager profiles as per your need.
The tool itself is a simpler version of HP ALM. You can create, manage and execute your test cases and optionally add any bug with it. The tool also has it’s own defect management module where you can log and track issues.
The tool also integrates with JIRA . Lets you sync all issue types and you can also map all your JIRA custom fields with this tool. It’s pretty hectic by luckily a one time effort though.
For me the dashboard is the best feature because it provides insights on your test cases, executions, trend analysis, and bug severity and status.
We also do our reporting through the tool as we can customize our test and bug reports in-app.
We used Testlink at my last company integrated with our bug tracking tool (Trac).
After the pain of moving projects from excel to Testlink the major pluses were:
Requirements capture
Stats for tests
Having change control on test cases
Being able to quickly produce new test plans for similar products
Being able to produce formal test reports fit for customer.
Pros:
Open source so easily modified
Cons:
Open source so don’t expect much from raising bug / enhancement requests
I just started using Testlink, being modifiable has never been a selling point, as a SDET, I have not really got time to spend reading someone else’s code when it’s not related to my day-job, but when it comes to being open, Excel is far more open in my opinion, Wayne.
I have always used in-house built management tools, because the source of truth today is my test scripts themselves that run once a day and once a week. An off-the-shelf package has never integrated with the real world well enough in any place I have worked to date. But, honestly, don’t use Excel.
Fortunately for us we had a dedicated test team so it was a project in its own right. It took at least about a month to integrate it with Trac (also modified) to our requirements so I’d not call this a “quick win”.
- Full treacability from bug to requirement
- Open source
- Supporting IOS, Android, MacOS, Windows, Linux
I just want a management tool, Testlink is free, but it’s looking like abandonware after 4 weeks of me trying to understand it’s “schema” or organization. Drawing Testlink reports is just bizarre or requires writing code just to do meaningfully. I’ve not got time to search for another tool, and am drifting back to Excel. We have 3+ testers, but developers and managers do testing too (everyone tests in concept), so a free tier just does not cut it, and an execution engine built into the tool is IMHO overkill and a distraction. Anyone else enjoying using Testlink http://forum.testlink.org/viewforum.php?f=11 who can give guidance, or offer low barrier alternative tools? Wary of SAAS or web hosted offerings that only want to sell you other features you don’t scale into.
Jira integration is nice, Automation integration is nice, but an import and export button with a web API behind them to to drive them fits the bill. For small companies like where I am now time and skills to be setting up a server is wasted.
- Jira integration
- Variety of test styles accommodated (not just stepped tests, like so many)
- Applicability to our internal process
We shouldn’t have to change process to handle the tool (although the right tool can help us develop our processes, for example the positive feedback styles handled by TestBuddy). As such we’re currently looking to move from Zephyr to Xray, which serves points 2 and 3 better.
2 Likes
In what ways did Zephyr not met your team requirements Stu? I am keen to just go with the 10-users tier and link Zephyr into our Jira somehow in a way that fits our process. Trials are so expensive when they require infrastructure and integration - even the time to just import all of your data. How can a tester trial all the tools out there and still do your day job? How do you find out about those hidden gem tools that almost nobody knows about that do a pretty good job, cheaply. I found another one that sounds good, I think was featured in a podcast last year https://testcaselab.com/ .
Deep breath…
Zephyr has seriously underwhelmed me over the last 4 years of use and this was as much a switch away from Zephyr as a switch towards something else. Zephyr have forced a very buggy “new” UI on users, only providing an opt-out weeks later, which meant we had to switch to a temporary solution (excel) mid-sprint for a period of weeks, about a year ago. This dazzling, unadvertised new UI failed to load all over the place, then did things like failing to save data, failed to record executions etc. It was a bloody nightmare for enterprise-level software, in use by a team building and testing enterprise-level software! Since then we’ve switched to the essentially deprecated and unsupported “legacy” UI (which still mostly works), although every time they push an upgrade it switches back to the new one, meaning more manual work for me to get to the functional one… our last year with Zephyr has been like pulling teeth!
As to the reasons for Xray, Zephyr (in this case Zephyr for Jira) is firmly rooted in the “stepped tests” mode of testing, whereas Xray (Xray Cloud) supports gherkin and exploratory styles natively. Zephyr is relatively expensive given we pay by Jira instance, for a company of 750+ Jira users… Xray bills the same but works out about half the price. Xray also stores all of it’s “nodes” as Jira issue types, meaning bulk operations, cloning etc work properly. Zephyr has a number of proprietary structures, most annoyingly test cycles. I have no way of cloning my regression pack without undertaking a day’s worth of donkey work - in Xray I can do this in 10 minutes.
May be too much detail… but I can’t warn you off Zephyr (for Jira) enough! But if you do go with it I hope you find it more suitable. We found Xray in a review, checked the Jira Marketplace reviews (top-rated TM tool) and have trialled it at https://sandbox.xpand-it.com/ 
2 Likes
This is very helpful , the cost of evaluating a tool is really high if you have not got 1 whole week to spend learning how to deploy a tool to start with. And that setup process can throw you off some of the more powerfull tools early on.
https://marketplace.atlassian.com/categories/test-management
1 Like
Ah my mistake, it was top when we began our plan to move though (about a year ago now!). Very surprised to see Zephyr rated so highly, perhaps it’s just us - although have spoken to testers in other orgs experiencing similar.
1 Like
I’ve yet to see a test management tool that works for me.
What I use is whatever the place I’m working for uses. The previous job had excel spreadsheets, then moved to TestLink. Neither fitted all that well, and nobody had adjusted to TestLink when I moved on. My current place I’m using TFS/VSTS/whatever for anything large, and the tasking in User stories/defects in Rally/Agile Central for smaller pieces.
My test cases are generally more along the line of a broad outline than a detailed test plan, and I haven’t got close enough to even think of using the ability to link a test case to a code routine.
In my opinion you use what you’ve got. If you’re picking something up, aim for something as close to your internal processes as you can get without blowing your budget to pieces, then make whatever adjustments you need to. The only time I’ve ever found the need to write detailed test cases was when someone who knew nothing about the software was going to be testing and I didn’t have the ability/time to mentor them.
1 Like
Git + Grep
You have simple and performative version control and search.
Anything else is bloated.
Not found a great off-the-shelf tool myself yet either 
I hate having huge dis-organised lists too. Have found infinitely more value in exploratory testing using the specs as a map, but when many teams have their own tester, having a common test spec language and organizing technique helps remove duplication.
I think this is the place anyone in a company that has staff churn, or has just joined, or is even in a start-up will find themselves in. Like when I went through manual release testing last week, I had completely forgotten to re-test the Raspberry Pi !! So having some kind of structure to the specific checks that really do have to be done, on all platforms - without too much duplication across teams happening does require a shared management tool of some kind.
Hello, Stu. My name is Laurent, I’m the VP of product management here at SmartBear and I’m sorry to hear that you and your teams had a negative experience with Zephyr for Jira.
When we acquired Zephyr in 2018, the company was working through a number of the issues you described, and since that time, we’ve made significant investments into strengthening integrations, usability, and adding new functionality that didn’t exist one year ago. We have introduced a new way to navigate in the application, increased performance, and released [Cucumber for Jira] that extends Zephyr with unique BDD capabilities like native Git integration and Living documentation.
I wish we could’ve helped you sooner, and if you would be interested in sharing any additional feedback with myself or our product team, I’d love to set up a conversation where we can learn more about what most impacted your experience.
1 Like
Hi Lauren.
I’m trying to stop using Testlink, since it’s just not powerfull enough without spending time coding, but Jira bug tracking plugins are expensive when you are already paying for Jira seats across a wide company who use it for things like marketing workflows etc. and where software engineering < half the seats. I can pay for my whole team pub lunches every Friday with the cost of most plugins when we look around. My other cost is that I would need to carve out a lot of time to import my data into Smartbear, and then run a whole release iteration - which will cost me 2 man-weeks, so there is significant commitment cost during the trial as well. Just to install the plugin and read the starting up guides I assume will take a day. So gentle start is a key factor for me. Still open to Smartbear if I can hear a testimonial from a happy tester in a <50 engineer company.
Personally I don’t care about integrations to BDD or TDD or whatever new fad is sweeping the industry. I want a test management tool that lets me mine my test specifications for relationships between, code (via well classified bugs and git commits) and a REAL USER workflow-based view of the world. Users don’t care about app architectures, and too many tools focus on testing the code behavior, instead of the how real customer use cases change over time. Too many models model the functionality, when we know that 80% of customers use 20% of your code, and that low attention span on some platforms mean traditional views of testing everything fail to quantify new risks.
Testlink does not let me easily export then just certain (filter) data, and then create mind-map kind of models and tag my specs. I want to group tests at will (let me add structured metadata) so that I can view for example all tests that contain the word “security”, or for all tests that happen to contain the word “mouse”, or “click”. Basically a tool that does one thing well, hold all of my data.
My pain with nice tool integrations, is that most of my career I have tested apps that require special frameworks, so although we use Appium here, that’s less than half of our test surface and a tenth of the risk area. Every job as a tester I’ve ever had did not favor off-the-shelf tooling, hence I’m looking at tools designed to be open. I spend enough time writing test framework code already to control environments which few of the frameworks do, well.
3 Likes