Testim.io - anyone using this tool?

🗄️ Archive
1

I just noticed the banner above and signed up for the free trial.

OK! it seems too good to be true and with 2 reviews that gave it 5 stars I am wondering if this tool has the potential to do what they say it does (so far its awesome). Why would anyone learn how to set up automation frameworks/coding etc.

I would really welcome opinions for both sides for AI automated tool - non tech ppl can use OR normal automation framework set up.

3 Likes
2

So from the recent 30 days of testing challenge, I’m wondering if @qakumarnz might be able to help you out. Ref:

Or perhaps @garydavidson83?

1 Like
3

Thanks Heather I will reach out to them both :+1:

1 Like
4

Hi there.

I had a couple of demos with testim and it seemed very impressive. In the end priorities changed and I had no time to carry on with the trial. I even had a 1 to 1 help session with a developer which was great even though I was on the free trial.

It had shown promise for a few ares of the application we have but there were also some limits to it. Sometimes using the scroll key on the mouse wouldn’t be picked up and elements were missed here and there. It was easy to fix some of these things but it is still a time intensive process.

I didn’t get to test the full powers that test.im advertises such as their machine learning so it would be interesting to hear how people get on with that.

The only real disadvantage I can see with it is that you don’t actually get to access any of the code. You can spend hours/days/weeks/months recording all the tests and fixing them etc but you are then left to the mercy of the website and their future ambitions. That is the only risk factor I can see in it. At least if you program it yourself you still have the code although this may also become worthless if the application goes through a major overhaul.

The other product someone recommended to me was https://www.leapwork.com/ which is a bit more hands on as it isn’t a recording based tool. it is visual though and requires no coding. Again my time was too limited to give it a real go.

Hope that helps.

3 Likes
5

Been using Testim.io for over 2 years now. They used to be tough on the code side but after their development in the last year I couldn’t be happier. Easy to integrate, easy to define tests, super robust.

2 Likes
6

Nice to read that as I have been playing with it for a few hours now (inbetween everything else I do) and it seems really easy to work with (at a basic level). Unfortunately our dev team wasn’t keen on the tool so there isn’t a budget for me to take it to the next level but I am going to continue plugging away with the free side. I’m determined to wow the team over what I believe is a pretty good tool for us at the stage our team is at. :+1:

7

Thanks Gary, interesting you came to that summation as I thought exactly the same thing about not having control over things especially if I am putting in serious hours to get something going.

I will have a look at leapwork

Cheers K

8

I’ve used Testim and had extensive demos and discussions with their team. It’s a really impressive and intuitive tool and I love using it.

However for use in our business it’s a hefty asking price with a not so simple pricing structure which has really prevented me from going further with them.

I’ve spent the last 6 months looking for a automated testing tool which can…

  • Quickly and intuitively create tests using click and record
  • Easily run large numbers of tests for multiple projects
  • Access sites behind my private network
  • Use AI/ML to automatically heal tests locators
  • Cost less than $300~ per month

Here’s a list of every product I’ve reviewed which didn’t meet all requirements:

Mabl, Ghost inspector, Boozang, CloudQA, testCraft, endtest, eggplant, Ranorex, Functionize and Testim.

Recently I’ve come across a product called TestSigma which has a lot of potential with new click record and AI/ML. I haven’t fully demoed the product but its looking positive and close to matching all my requirements.

2 Likes
9

Hi Kimberley,

I have been using Testim extensively for two months and here are some of my thoughts.

My Background & Getting Started with Testim:
I am a college student who has limited coding (basic python) and no test automation experience.

For my summer internship, my company wanted to explore Testim to automate some of the QA process for Salesforce development work. It took me about a week to get the basics of the tool and three more weeks to fully understand the more advance functions (test parameterization, JavaScript custom action etc). I created a total of 36 parameterized tests cases during my two-month internship. Our test scenarios were more complicated so the number could be a lot larger depending of the complexity of your test scenarios.

Overall, I thought the tool was easy to use and quite powerful. I was able to create test really quickly as the tool works almost as if you are doing screen recording. The amount it takes to click through your process you wanted to test is generally how long it takes to create the test. However, the most time consuming part was after the initial recording, when you have to go back and make sure your process was recorded appropriately.

The built-in functionalities of the tool are very powerful. You can validate for almost any element / text on the screen, and when you can’t, you can always create simple JavaScripts codes to help you identify the element.

In addition to using Testim for quality assurance testing, we also took advantage of the tools ability to create test data. For our testing purpose, we used to have to manually create 20 - 30 test orders in Salesforce. Now with the automated tests, we just need to update our test parameters and Testim automatically creates our test data as it runs the tests.

Pros:

  • Easy to learn, no coding required
  • Stable test runs
  • Quick test creation
  • The ability to recognize and adapt to layout changes

Areas of Improvement:

  • Better help documentationThe help documents on the website are great source of learning. They cover all the general topics. However, I found that sometimes the articles could small details that are key to making the tests work. I wish there were step-by-step instructions with some of the articles. Maybe also a Q&A of common user questions.
  • A Lack of CommunityThe Live Chat function was very helpful. However, there is limited source of help/community beside the live-chat and the help document center. It would be nice to have a community where people share their insights and problems.
  • Test Organization/ Export Data Although the tests are organized beautifully on the UI, it would be nice if I could export a list of tests and test parameters in Excel format.

Hope this helps!

5 Likes
10

Hi May,

Thank you so much for all the info. I agree its a great tool but for our smallish start-up the price factor was an issue so the team decided against it. I think your results speak for themselves on this tool success.

11

This tool is super easy to use while working with functional testing services and the best part is you don’t have to get into the code for working with it. This tool is basically good for executing and maintaining the automated test cases. It records the data with every action we perform and later we can save them & execute. Following are the main features of Testim tool:

  1. It creates stable automated test cases and execute them with less hassle.
  2. The created test cases can be ran on multiple browsers say Chrome, Firefox, Safari, IE and Edge.
  3. We can create tests by either clicking through our app or using the custom code.
  4. It comes as an extension with Chrome browser thus making it really easy to install.
  5. The tests we are saving with it are saved directly to the cloud.
  6. It records the steps performed with screenshots.
  7. User can add the step in between the recorded steps and can also delete & update the existing steps.
  8. User can also add random data for the recorded test steps in order to avoid the duplicacy.
  9. A green sign is appeared on every block that is executed successfully while running the saved test.
  10. The created Test case can be saved and can be executed anytime.

There are few Pros and cons of Testim:

Pros:

  • Saves time
  • Have good test stability.
  • Easy to use.

Cons:

  • Difficulties with image verification.
  • It is a paid tool.

In conclusion, Testim is a good tool that a person who does have not much knowledge about automation can use. Hope this information is helpful for you.

5 Likes
12

Hi Kim,
Could you give us some info on the price range you had to pay?
The web site pricing page mentions only “free” and “contact us”…

1 Like
13

I think they give you a price quote per requirements.

We decided as a team not to go with this tool for many reasons but I will say the people I chatted to at Testim.io were very helpful and courteous. Their support team were well trained and even though there was a significant time difference they got back to me very quickly.

Give it a go and see if it suits your teams needs.

Good luck :+1:

3 Likes
14

So I have been looking into a range of tools for the regression component of our automation suite over the past 12 months.

A few things to note before you read on -
We are a small organisation with only 1 tester dedicated to do the BAU work, project work and manage all the automation suite. We do have an additional 2 testers on contract but this is due to a large number of projects happening for 2 years.

So initially we did use Selenium however the upkeep became a nightmare due to the fact a single new web page added into a workflow can break many tests and requires a significant amount of time to upkeep. In addition, we ultimately cannot afford to expend the time manually running tests and then coding the tests.

We then decided we would look to UI record and playback type tools and here is our findings -

Test Plant: Steep learning curve, great reporting options, support was not very responsive and it is expensive at $40k++ a year.
Tricentis Tosca: Great tool, easy to maintain, great reporting options and has some great features like their test matrix that can turn one form style test into 100 tests covering all variables. Cost is extreme for a small organisation - $50k++ a year.
Testim: Great tool and easy to use for basic functions however does require some time to create complex items such as grab text off the UI in Test A and reuse in Test D. Their support team is the best I have seen however the reporting options of Testim are very limited. I have placed a request to get reporting improved.

I am also looking into Test Sigma as an additional option now.

In summary,

I might buy and use Tricentis Tosca if I could afford it and had a very big team. However, I never got to check out their support abilities…

I love the time saved using Testim record options - this reduction in the need to manually run and then code means that coding time is now free to do maintenance or create more tests.
I also love their support - I honestly was stunned with the quality of their support team. As a small team it is definitely leading my regression choice. If I get it for a decent price and I get the reporting I need then I doubt I will choose any other regression tool.

Now I am off to trial TestSigma…

1 Like
15

Gary & Kimberly, this Dojo post might interest you re: LeapWork (formerly LeapTest):


#selfishplug #plugfortheDojo

16

Thank you @jesper I will def. read it and have a look.

I’m just starting to use the updated selenium IDE and I can see I will need some better coding skills in ruby to get what I actually want it to test which is really just the UI & UX flow as our devs write automated tests for all the integration testing and units.

Yes the issue I have is $$$ and some push back from some key team members on me learning to automate things. :frowning: … oh well its a new year soon and maybe everyone will just need to get comfortable with testing moving forward.

1 Like
17

Nice summary @bookman

I am in a similar situation to yourself and $$$ really count so the stakeholders need to see a return for the investment naturally.

Agree with Testim support team they are really amazing considering the time difference between them and myself.

I am diving into Selenium starting with the upgraded IDE and hoping to learn enough ruby to pull this off with what I need which ultimately is only UI & UX testing with element verification.

I am very keen to see what you think of TestSigma.

Cheers k

2 Likes
18

Thanks for this detailed feedback!

19

I have been evaluating Testim for some time and I do have some concerns.

The UX is not great for organizing test cases and test suites. It looks cool and smooth in a demo with a few test cases, but when you have hundreds or thousands of test cases, it gets pretty bad.
If you have long test cases (which you really shouldn’t), the UI does not handle it well either.

If you ever have to cancel Testim, you can get the code out, but it’s probably going to be in garbage form and will have to be rewritten anyway.

1 Like
20

Our company has been using Testim for over a year and we find it to be very easy and efficient.

Prior, we had been using other tools that were not as productive. Testim is able to run multiple tests for different projects and can easily create new tests using click and record. With Testim, we are able to schedule tests that run automatically, at any time.

One aspect that we particularly love about Testim is that when a test fails an email is sent with the description of the error and a screenshot of the page. With Testim, we are also able to customize our test based on browser, OS, URL and run tests under a variety of conditions. This helps to save time as we work with many clients in a fast paced work environment.

2 Likes