What makes a tool a good tool?

I am continuing to build our automation certification and I have another question about tools that I’d love to get your thoughts on:

What makes a tool a good tool?

Specifically, how do you evaluate a tool and work out if it’s useful or not to you. I’m sure there are lots of different ways in which tools can be evaluated so we’d love to know what criteria do you have for tools.

As always, thank you for your contribution :robot:


Doing one small, thing extremely well and playing well with other tools.


Hi @mwinteringham , I went back to my slides when i had to select a whole set of tools for Test & Defect Management, Performance Test , Mobile Automation and below are some of the things that i had used:

Some of the key considerations were :
• Budget available
• Number of Users
• Support to my current technology stack
• Current tool stack in the organisation
• Licensing model - Concurrent or node based
• Overall features and functionality
• Ease of use
• Integration with existing tools
• Integration with Jira
• Ease of configuration and maintenance
• Amount of training required
• Reporting capabilities
• Cloud capabilities
• Future proof
• Requirement Traceability capabilities
• Gartner Quadrant position
• Other functionalities that the tool provides
• Support and Training Availability
• Background of the company and their seriousness of their further investments to keep up to the latest trends
• Frequency of Releases

@mwinteringham - let me know if this helps and also if i have missed anything key. The above are in no specific order, but all are important, some more than the other.


In the threads that you keep opening, you refer to the ‘new automation learning journeys’, but the questions are put in a generic manner ‘a good tool’. Are you asking the questions in the context of automation, testing, or personal/company use?

I’d say it depends:

  • Sometimes it has to be quick to find when searching the internet for specific keywords, set up, and start doing the work.
  • Sometimes it has to be flexible, portable, easy to migrate, integrate, etc…
  • Sometimes it’s about the speed of getting something done and because of experience and a setup already done, you jump on an existing tool.
  • Sometimes it has to be free, other times it has to be paid for SLA.
  • Sometimes it has to be a cheap cost of usage overall;
  • Sometimes it has to be ‘precooked’ that you just open, drag-drop, or click-tap, and it’s done;
  • Sometimes it has to be usable also by business or understood by managers so they want the tool to have a human-readable interface,
  • Sometimes you don’t care, as you prefer to pay a consultancy 100k to do the X stuff in 3 months;
  • Sometimes the company pays a consultancy to do slides of comparison between tools with strengths they think matter;
  • One time I had 4 managers in a consultant’s presentation on tools asking, but what if we want to use this framework to do also A, and B, and C, and D, and maybe also make the X subsidiary company use it, and P department.

And as a tester, I use different types of tools so I want the tool to fit in my context of use:

  • Product, project, and issue management
  • Systems, OS, interfaces
  • IDE and code editors
  • Programming or scripting
  • DevOps and Code management
  • Release, and deployment management
  • Database
  • API
  • Logging, stats, and data analytics
  • Note-taking, and structuring info
  • Testing helpers
  • Scanners
  • Debugging and pinpointing bugs
  • Custom Platforms
  • Automation in testing

Then it matters also:

  • in what cultural context you use a tool,
  • which part of the world,
  • what knowledge it is around for using a tool
  • what business you’re in,
  • what product is built;
  • in what phase you are in a project/product;
  • what role do you have;
  • what managers do you have, decision-makers, stakeholders;
  • company size, state-owned or not, regulated or free;

I think the answer doesn’t lie in, it has to do this or that but more in a way of “The tools fits the needs of my current project”.

The most obvious criteria’s are already mentioned so I’m not going to repeat it but something that is often overlooked is: Upskilling of employees.

You can have a kick-ass tool which fits all of your needs but if you only have JavaScript programmers who dislike Java, you can’t introduce a Java-Based framework.

Who is going to use the tool? Do we need to support multiple roles in the tool? (Like a security officer, or view only on dashboards or … )

When looking for a new tool, you also have to view at the commitment of your employees towards the tool. Are they backing you up on this tool or not? Because if one of them holds a grudge on a tool, he’s likely to less use it.

Is the tool scalable towards other teams if my project kicks off and people become jealous? :slight_smile:


Some really thorough answers here.

A few random thoughts/considerations:-

  • metrics aren’t everything, but gathering data to inform your views and remaining objective is important - what tasks do teams need to do that isn’t being met by current tools? What is the common consensus/preference? I’ve seen the one person get tempted by the shiny new thing only to end up forcing it through and not taking the team with them. It didn’t work out and ended up being mothballed.

  • I’d also want to know how easy it is to switch in and especially out of this tool if it isn’t meeting your needs in future. There is a trend for vendors to get you hooked on a fantastic product for free, only to get all Ryanair and start introducing charges when they know you are dependent on the product and its too expensive/difficult to move.

  • what is the community support like? Is it going to be easy to train or hire people to work with this tool, or is this tool going to be seen as an unpopular CV killer?

  • how long do you intend the tool to be used? How often will you review alternatives or ensure the portfolio you have is working out?

  • if people have difficulties using this tool, who is going to support them? Ideally this will be someone internal as well as any vendor support. How about training for new features, or central licence/configuration?


You are right and this is very important. In fact I worked along with my team who helped in evaluating the functionalities, ease of use, training required, etc. We then debated the pros and cons and finally put in our recommendation. Also we did an DAR (Decision Analysis and Resolution) excercise to validate our recommendation.


Very valid points to consider @bethtesterleeds . Thanks for that.

1 Like

Wowser! Some really excellent thoughts here, thank you so much for all of them. We’ll be selecting bits of your feedback and adding them into our learning journey to give some perspectives outside of mine and Richards.

I would like to add another point to bethtesterleeds’s response. The following comes after the purchase/decision is made, but also an important factor for success (what makes the tool good tool)

  • Training and onboarding:
    • Well, some tools are easy to learn, and some are harder.
    • As an organization leader, if you are bringing a new tool, it needs the support and love by all. This is the absolute must for success, or else people start blaming the tool/vendor.
    • A change is harder, but good understanding by all can make it easier for everyone. Start putting active sessions so everyone in the org understands it.
    • As a leader, track adoption, and push for it usage in every relevant meeting.
1 Like

Here’s my list:

  • Price (does it fit in my budget)
  • Does it get the job done (aka ‘how successful was the timeboxed session?’/ ‘how well does it match my tool selection criteria?’)
  • Existing knowledge in the team
  • Willingness of the team to learn the tool
  • Integration with the current tooling (dev tech stack, CI/CD, Test Management Tool, etc.)
  • Reporting capabilities (again: Integration with existing tooling/ reporting)
  • Documentation
  • Community/ vendor support: How good is it? How reactive?
  • Ease of use/ usability
  • Can I live with the disadvantages/ shortcomings?
  • What do colleagues/ friends/ my network have to say about the tool? What information can I get from my other ‘sources of information’

Ability to use it ‘almost’ with your eyes shut.


What makes a car a good car …?

1 Like

It does the job you need it to do and does that job well.

ah, it does not need regular repairs, blooming well said!

I’m trying to unwrap this one.
A car can facilitate in some cases the commuting of some person from one point to the other most usually at a desired time, but with some major downsides: money, pollution, health(not cycling or walking, increasing inhalation of toxic gases), increased risk of accidents for driver or other traffic participants, decrease of green space, and others.

A car is good if it’s highly reliable for the initial purpose and reduces the other major downsides.

don’t take the car analogy too far @ipstefan :slight_smile: ha ha

For me the biggest disappointment is when you later on find that the tool has some dependency that means you cannot use it in a scenario, so for me a simple tool that does not hide too much information, but that expose just the right amount, those are the kinds of tools that are easy to use as building blocks or jigs and also far less likely to break down at a future point.

Example: when someone changes the minimum crypto algorithm, that means you have to for example upgrade and rebuild the tool for SSL 3.X , it’s always good to know just enough about tool dependencies. And thus not have to keep doing running repairs.

Both the tool and the car case show that there is a difference between what can be considered good in general and what is good for a particular situation. What is good in general may not be good for me and what is not good in general may be perfect for me. Mark’s question is about determining its suitability for a particular context. So it does not matter (much) here what Gartner says about Selenium or Robot framework or Tosca.

We have been listing various characteristics, or rather questions to ask, but the more of these are listed the more obvious the lack of structure becomes. I am working on structuring them so that they become easier to work with. I’ll be back (spoken with Arnold’s accent, of course).

It may well be that by ignoring tool price, we might create a more level playfield @martingijsen . Because free closes the door on having that “top tier” of actual high-quality and perfection for people to aim for when buying, at the same time.

Tool price is just one facet of cost, and cost is just one aspect to consider. Important for some, less so in other contexts.

1 Like