I am continuing to build our automation certification and I have another question about tools that I’d love to get your thoughts on:
What makes a tool a good tool?
Specifically, how do you evaluate a tool and work out if it’s useful or not to you. I’m sure there are lots of different ways in which tools can be evaluated so we’d love to know what criteria do you have for tools.
As always, thank you for your contribution 
Doing one small, thing extremely well and playing well with other tools.
Hi @mwinteringham , I went back to my slides when i had to select a whole set of tools for Test & Defect Management, Performance Test , Mobile Automation and below are some of the things that i had used:
Some of the key considerations were :
• Budget available
• Number of Users
• Support to my current technology stack
• Current tool stack in the organisation
• Licensing model - Concurrent or node based
• Overall features and functionality
• Ease of use
• Integration with existing tools
• Integration with Jira
• Ease of configuration and maintenance
• Amount of training required
• Reporting capabilities
• Cloud capabilities
• Future proof
• Requirement Traceability capabilities
• Gartner Quadrant position
• Other functionalities that the tool provides
• Support and Training Availability
• Background of the company and their seriousness of their further investments to keep up to the latest trends
• Frequency of Releases
@mwinteringham - let me know if this helps and also if i have missed anything key. The above are in no specific order, but all are important, some more than the other.
In the threads that you keep opening, you refer to the ‘new automation learning journeys’, but the questions are put in a generic manner ‘a good tool’. Are you asking the questions in the context of automation, testing, or personal/company use?
I’d say it depends:
And as a tester, I use different types of tools so I want the tool to fit in my context of use:
Then it matters also:
I think the answer doesn’t lie in, it has to do this or that but more in a way of “The tools fits the needs of my current project”.
The most obvious criteria’s are already mentioned so I’m not going to repeat it but something that is often overlooked is: Upskilling of employees.
You can have a kick-ass tool which fits all of your needs but if you only have JavaScript programmers who dislike Java, you can’t introduce a Java-Based framework.
Who is going to use the tool? Do we need to support multiple roles in the tool? (Like a security officer, or view only on dashboards or … )
When looking for a new tool, you also have to view at the commitment of your employees towards the tool. Are they backing you up on this tool or not? Because if one of them holds a grudge on a tool, he’s likely to less use it.
Is the tool scalable towards other teams if my project kicks off and people become jealous? 
Some really thorough answers here.
A few random thoughts/considerations:-
metrics aren’t everything, but gathering data to inform your views and remaining objective is important - what tasks do teams need to do that isn’t being met by current tools? What is the common consensus/preference? I’ve seen the one person get tempted by the shiny new thing only to end up forcing it through and not taking the team with them. It didn’t work out and ended up being mothballed.
I’d also want to know how easy it is to switch in and especially out of this tool if it isn’t meeting your needs in future. There is a trend for vendors to get you hooked on a fantastic product for free, only to get all Ryanair and start introducing charges when they know you are dependent on the product and its too expensive/difficult to move.
what is the community support like? Is it going to be easy to train or hire people to work with this tool, or is this tool going to be seen as an unpopular CV killer?
how long do you intend the tool to be used? How often will you review alternatives or ensure the portfolio you have is working out?
if people have difficulties using this tool, who is going to support them? Ideally this will be someone internal as well as any vendor support. How about training for new features, or central licence/configuration?
You are right and this is very important. In fact I worked along with my team who helped in evaluating the functionalities, ease of use, training required, etc. We then debated the pros and cons and finally put in our recommendation. Also we did an DAR (Decision Analysis and Resolution) excercise to validate our recommendation.
Wowser! Some really excellent thoughts here, thank you so much for all of them. We’ll be selecting bits of your feedback and adding them into our learning journey to give some perspectives outside of mine and Richards.
I would like to add another point to bethtesterleeds’s response. The following comes after the purchase/decision is made, but also an important factor for success (what makes the tool good tool)
Here’s my list:
Ability to use it ‘almost’ with your eyes shut.
What makes a car a good car …?
It does the job you need it to do and does that job well.
ah, it does not need regular repairs, blooming well said!
Hmmm,
I’m trying to unwrap this one.
A car can facilitate in some cases the commuting of some person from one point to the other most usually at a desired time, but with some major downsides: money, pollution, health(not cycling or walking, increasing inhalation of toxic gases), increased risk of accidents for driver or other traffic participants, decrease of green space, and others.
A car is good if it’s highly reliable for the initial purpose and reduces the other major downsides.
don’t take the car analogy too far @ipstefan ha ha
For me the biggest disappointment is when you later on find that the tool has some dependency that means you cannot use it in a scenario, so for me a simple tool that does not hide too much information, but that expose just the right amount, those are the kinds of tools that are easy to use as building blocks or jigs and also far less likely to break down at a future point.
Example: when someone changes the minimum crypto algorithm, that means you have to for example upgrade and rebuild the tool for SSL 3.X , it’s always good to know just enough about tool dependencies. And thus not have to keep doing running repairs.
Both the tool and the car case show that there is a difference between what can be considered good in general and what is good for a particular situation. What is good in general may not be good for me and what is not good in general may be perfect for me. Mark’s question is about determining its suitability for a particular context. So it does not matter (much) here what Gartner says about Selenium or Robot framework or Tosca.
We have been listing various characteristics, or rather questions to ask, but the more of these are listed the more obvious the lack of structure becomes. I am working on structuring them so that they become easier to work with. I’ll be back (spoken with Arnold’s accent, of course).
It may well be that by ignoring tool price, we might create a more level playfield @martingijsen . Because free closes the door on having that “top tier” of actual high-quality and perfection for people to aim for when buying, at the same time.
Tool price is just one facet of cost, and cost is just one aspect to consider. Important for some, less so in other contexts.