So I have been looking into a range of tools for the regression component of our automation suite over the past 12 months.
A few things to note before you read on -
We are a small organisation with only 1 tester dedicated to do the BAU work, project work and manage all the automation suite. We do have an additional 2 testers on contract but this is due to a large number of projects happening for 2 years.
So initially we did use Selenium however the upkeep became a nightmare due to the fact a single new web page added into a workflow can break many tests and requires a significant amount of time to upkeep. In addition, we ultimately cannot afford to expend the time manually running tests and then coding the tests.
We then decided we would look to UI record and playback type tools and here is our findings -
Test Plant: Steep learning curve, great reporting options, support was not very responsive and it is expensive at $40k++ a year.
Tricentis Tosca: Great tool, easy to maintain, great reporting options and has some great features like their test matrix that can turn one form style test into 100 tests covering all variables. Cost is extreme for a small organisation - $50k++ a year.
Testim: Great tool and easy to use for basic functions however does require some time to create complex items such as grab text off the UI in Test A and reuse in Test D. Their support team is the best I have seen however the reporting options of Testim are very limited. I have placed a request to get reporting improved.
I am also looking into Test Sigma as an additional option now.
I might buy and use Tricentis Tosca if I could afford it and had a very big team. However, I never got to check out their support abilities…
I love the time saved using Testim record options - this reduction in the need to manually run and then code means that coding time is now free to do maintenance or create more tests.
I also love their support - I honestly was stunned with the quality of their support team. As a small team it is definitely leading my regression choice. If I get it for a decent price and I get the reporting I need then I doubt I will choose any other regression tool.
Now I am off to trial TestSigma…