I’m searching for a testing management tool which is listed as “the best one”. I need your advices and your thoughts about the tool you are using. And would you like to answer this question: " Which features of a test management tool that make you choose/use them?" Thank you guys!
There is no “best one”, but there is probably a best fit for your situation.
Some projects I do involve a range of business users doing testing, here some tools shine with features like mobile versions and automatic screen capture.
Some projects I do involve large code bases in Visual Studio and agile interactions with the developers. Here linking work items together is important.
there is TestRail, VSTS, Panaya, Plutora, Tricentis Tosca and many more…
Thanks for your useful info. And can I ask you a question? Which feature of a test management tool that make use it?
sure I prefer tools that have a requirements module (or integrations to the same), a hierarchy of test activities, tests (scripts, cases… YMMV) and bugs (or a reference to bug tracker). Most of the above tools have this.
But what make me choose the final tool is a matter of project context.
We have been using Kualitee for our test & bug management needs for quite some time now. The tool supports both manual and Automation testing. It has integration with Selenium for test automation and Jenkins & Bitbucket pipelines for release management.
The tool is also free for a small team of 3 people or add up to 10 user for $45. You can create tester, developer and QA manager profiles as per your need.
The tool itself is a simpler version of HP ALM. You can create, manage and execute your test cases and optionally add any bug with it. The tool also has it’s own defect management module where you can log and track issues.
The tool also integrates with JIRA . Lets you sync all issue types and you can also map all your JIRA custom fields with this tool. It’s pretty hectic by luckily a one time effort though.
For me the dashboard is the best feature because it provides insights on your test cases, executions, trend analysis, and bug severity and status.
We also do our reporting through the tool as we can customize our test and bug reports in-app.
We used Testlink at my last company integrated with our bug tracking tool (Trac).
After the pain of moving projects from excel to Testlink the major pluses were:
Stats for tests
Having change control on test cases
Being able to quickly produce new test plans for similar products
Being able to produce formal test reports fit for customer.
Open source so easily modified
Open source so don’t expect much from raising bug / enhancement requests
I just started using Testlink, being modifiable has never been a selling point, as a SDET, I have not really got time to spend reading someone else’s code when it’s not related to my day-job, but when it comes to being open, Excel is far more open in my opinion, Wayne.
I have always used in-house built management tools, because the source of truth today is my test scripts themselves that run once a day and once a week. An off-the-shelf package has never integrated with the real world well enough in any place I have worked to date. But, honestly, don’t use Excel.
Fortunately for us we had a dedicated test team so it was a project in its own right. It took at least about a month to integrate it with Trac (also modified) to our requirements so I’d not call this a “quick win”.
- Full treacability from bug to requirement
- Open source
- Supporting IOS, Android, MacOS, Windows, Linux
I just want a management tool, Testlink is free, but it’s looking like abandonware after 4 weeks of me trying to understand it’s “schema” or organization. Drawing Testlink reports is just bizarre or requires writing code just to do meaningfully. I’ve not got time to search for another tool, and am drifting back to Excel. We have 3+ testers, but developers and managers do testing too (everyone tests in concept), so a free tier just does not cut it, and an execution engine built into the tool is IMHO overkill and a distraction. Anyone else enjoying using Testlink http://forum.testlink.org/viewforum.php?f=11 who can give guidance, or offer low barrier alternative tools? Wary of SAAS or web hosted offerings that only want to sell you other features you don’t scale into.
Jira integration is nice, Automation integration is nice, but an import and export button with a web API behind them to to drive them fits the bill. For small companies like where I am now time and skills to be setting up a server is wasted.
- Jira integration
- Variety of test styles accommodated (not just stepped tests, like so many)
- Applicability to our internal process
We shouldn’t have to change process to handle the tool (although the right tool can help us develop our processes, for example the positive feedback styles handled by TestBuddy). As such we’re currently looking to move from Zephyr to Xray, which serves points 2 and 3 better.
In what ways did Zephyr not met your team requirements Stu? I am keen to just go with the 10-users tier and link Zephyr into our Jira somehow in a way that fits our process. Trials are so expensive when they require infrastructure and integration - even the time to just import all of your data. How can a tester trial all the tools out there and still do your day job? How do you find out about those hidden gem tools that almost nobody knows about that do a pretty good job, cheaply. I found another one that sounds good, I think was featured in a podcast last year https://testcaselab.com/ .
Zephyr has seriously underwhelmed me over the last 4 years of use and this was as much a switch away from Zephyr as a switch towards something else. Zephyr have forced a very buggy “new” UI on users, only providing an opt-out weeks later, which meant we had to switch to a temporary solution (excel) mid-sprint for a period of weeks, about a year ago. This dazzling, unadvertised new UI failed to load all over the place, then did things like failing to save data, failed to record executions etc. It was a bloody nightmare for enterprise-level software, in use by a team building and testing enterprise-level software! Since then we’ve switched to the essentially deprecated and unsupported “legacy” UI (which still mostly works), although every time they push an upgrade it switches back to the new one, meaning more manual work for me to get to the functional one… our last year with Zephyr has been like pulling teeth!
As to the reasons for Xray, Zephyr (in this case Zephyr for Jira) is firmly rooted in the “stepped tests” mode of testing, whereas Xray (Xray Cloud) supports gherkin and exploratory styles natively. Zephyr is relatively expensive given we pay by Jira instance, for a company of 750+ Jira users… Xray bills the same but works out about half the price. Xray also stores all of it’s “nodes” as Jira issue types, meaning bulk operations, cloning etc work properly. Zephyr has a number of proprietary structures, most annoyingly test cycles. I have no way of cloning my regression pack without undertaking a day’s worth of donkey work - in Xray I can do this in 10 minutes.
May be too much detail… but I can’t warn you off Zephyr (for Jira) enough! But if you do go with it I hope you find it more suitable. We found Xray in a review, checked the Jira Marketplace reviews (top-rated TM tool) and have trialled it at https://sandbox.xpand-it.com/
This is very helpful , the cost of evaluating a tool is really high if you have not got 1 whole week to spend learning how to deploy a tool to start with. And that setup process can throw you off some of the more powerfull tools early on.
Ah my mistake, it was top when we began our plan to move though (about a year ago now!). Very surprised to see Zephyr rated so highly, perhaps it’s just us - although have spoken to testers in other orgs experiencing similar.
I’ve yet to see a test management tool that works for me.
What I use is whatever the place I’m working for uses. The previous job had excel spreadsheets, then moved to TestLink. Neither fitted all that well, and nobody had adjusted to TestLink when I moved on. My current place I’m using TFS/VSTS/whatever for anything large, and the tasking in User stories/defects in Rally/Agile Central for smaller pieces.
My test cases are generally more along the line of a broad outline than a detailed test plan, and I haven’t got close enough to even think of using the ability to link a test case to a code routine.
In my opinion you use what you’ve got. If you’re picking something up, aim for something as close to your internal processes as you can get without blowing your budget to pieces, then make whatever adjustments you need to. The only time I’ve ever found the need to write detailed test cases was when someone who knew nothing about the software was going to be testing and I didn’t have the ability/time to mentor them.
Git + Grep
You have simple and performative version control and search.
Anything else is bloated.
Not found a great off-the-shelf tool myself yet either
I hate having huge dis-organised lists too. Have found infinitely more value in exploratory testing using the specs as a map, but when many teams have their own tester, having a common test spec language and organizing technique helps remove duplication.
I think this is the place anyone in a company that has staff churn, or has just joined, or is even in a start-up will find themselves in. Like when I went through manual release testing last week, I had completely forgotten to re-test the Raspberry Pi !! So having some kind of structure to the specific checks that really do have to be done, on all platforms - without too much duplication across teams happening does require a shared management tool of some kind.