Comparison between Azure DevOps and Testrail

What are the major differences between Testrail and Azure DevOps for test case management?

So far, the only function that is not in azure that is in testrail appears to be the import function, that would be critical for us. However, it appears Azure’s shared test steps are a function that is not in testrail.

They appear to be pretty comparable in terms of reporting.

What’s been your experience?

I’d be curious about this myself - I just started on with a team with practically no QA process so am working on setting up all the things. They are heavily invested in Azure DevOps and would prefer to use those products but are ultimately willing to consider what I think is best. I’ve not used it in the past (Azure DevOps) and am not finding a lot about what people like or don’t about it. Any input to share thus far?

I can’t say anything about TestRail, but Azure DevOps = Team Foundation Server Online with extra goodies.

Probably the biggest benefit to Azure DevOps is the integration with All The Things. As long as it’s set up correctly, you can connect your test automation to test cases, your test cases to work items (regardless of the schema you’re using for the project - that affects whether it’s defects, user stories, bugs, or something else), and hook it all to an Azure Devops build/release pipeline.

Manual test cases can be as detailed or broad as you please - I tend to keep mine broad and use the test steps to cover different settings that a test case needs to cover. You can do anything from stepping through your test case using the web tool to just marking it pass/failed when you’re done testing.

There’s also a ton of plug-ins/extensions, many of them free, that you can use - including a plug-in for Firefox and Chrome that allows you to record exploratory sessions and log them to Azure DevOps, associated with the user story/bug/whatever that you need them linked to.

Honestly, there’s so much there it can be overwhelming at first - it took me months to get my first automated test pipeline working. Now I can set one up in a day or so and get emailed the results.

1 Like

HA, re: overwhelming… YES. I’m SO there. I’ve not worked with it before and am definitely trying to sort out what the workflow should look like. Questions for you since it sounds like you’ve worked with it a fair amount… do you know if there’s a way to automatically add a group of test cases (a test suite) into the next iteration? For example, I have a set of tests I’d like run each sprint. If so, how can I accomplish this? Thanks!