Success in any test automation depends on identifying the right tool for automation. Selecting the “correct” Testing Tool for your project is one of the best ways to achieve the project target.
Step 1) Identify the requirement for tools
How can you select a testing tool if you do not know what you are looking for?
Step 2) Evaluate the tools and vendors
Analyze the commercial and open source tools that are available in the market, based on the project requirement.Evaluate the quality of the tool by taking the trial usage & launching a pilot.
Step 3) Estimate cost and benefit
A cost-benefit analysis should be performed before acquiring or building a tool.
Step 4) Make the final decision
Have a strong awareness of the tool. It means you must understand which is the strong points and the weak points of the tool. Balance cost and benefit.
Your decision may adversely impact the project, the testing process, and the business goals; you should spend a good time to think hard about it.
Selecting the right tool can be a tricky task. The following criteria will help you select the best tool for your requirements:
Environment Support
Ease of use
Testing of Database
Object identification
Image Testing
Error Recovery Testing
Object Mapping
Scripting Language Used
Support for various types of test – including functional, test management, mobile, etc…
Support for multiple testing frameworks
Easy to debug the automation software scripts
Ability to recognize objects in any environment
Extensive test reports and results
Minimize training cost of selected tools
Tool selection is one of biggest challenges to be tackled before starting automation. First, identify the requirements, explore various tools and their capabilities, set the expectation from the tool and go for a Proof Of Concept.
Type of testing: Functional, performance, load test…
Generate the test result in the desired format
Generate the test cases automatically
Easy to develop and maintenance scripts
…
Step 2) Evaluate the tools and vendors that are available in the market, based on the project requirements.
Step 3) Estimate cost and benefit
Step 4) Make the final decision
Since I am new to automation, and I am not really in a position to purchase software, the short answer is "Whatever I can get my hands on, or whatever I can build."It’s probably no coincidence that Selenium is popular and also free, as is Python, as is my new favorite tool ‘AutoiT.’
If I was in a position to get any tool I wanted? Then obviously I would do a lot of research, and THAT would mean asking my fellow testers here at the ministry. That’s right! This has turned into a commercial for ministryoftesting.com. Ministry of testing - for all your testing tool related questions! I am not a paid shill!
What are your testing requirements?
It is highly required to have a deep understanding of the application being tested and its technologies involved.
2.How complex is your application testing requirement?
Sometimes, there might be projects where you only need to perform load testing or most of the time, there will be requirements of only web testing or mobile testing or web services testing. So, you need to have a clarity in setting the criteria for selecting the automation testing tool.
Do you have skilled resources?
You have prepared the documents that have the details of the application testing requirements. Well, now it’s time to find out whether your organization has enough skilled resources to handle that particular testing project as well as the new testing tool.
How much you want to invest?
When working on finalizing the budget to purchase the testing tool, you also need to make sure, you have the prices of all the tools available on hand.See, before experimenting, you’re not going to finalize the tool. However, to get the budget approved by the top management, you will require the prices of the overall software test automation tools available in the industry.
I wasn’t the one who decided the tool, but I guess we went by “popularity” thinking if a tool is popular enough, it would cater to the basic needs of automation anyway! Could be a bad approach but many small scale companies would do that.
How or which tools - depends upon compatibility-with the design application, functionality-has all the required features for automation, usability-easy to use, maintainability-support for the tool, flexibility-licensing & affordability-cost.
In all my years in QA, I can honestly say this topic is near and dear to my heart. It was the subject of many conversations I had with my QA manager(s), and the time invested was time well spent, because one of the many lessons learned was that not every tool fits the project you are working with, and custom-codes does not play nice with out-of-the box solutions. Selenium was (and still is) the reliable go-to source, but to choose what tool to use required extensive thought and careful research.
Below is a list of what we used to select the tool for the job:
1 Ease-of-use
Is simple, cost-effective, and easy to set up, requiring minimal effort outside of the most basic environmental preparation.
Allows tests to be written quickly and efficiently, even by a novice user.
Allows tests to be maintained effortlessly.
Should not require a dedicated person to use.
2 Development Language
Allow tests to be written in a language familiar to the team, thus requiring minimal learning curve or advanced knowledge.
Has the flexibility to allow use of external libraries and modules as necessary.
3 Desktop Support
Can run tests across all browsers in scope.
4 Mobile Support
Can run tests on all mobile devices in scope.
5 Web Service (API) Support
Able to create tests for APIs, web services, and the like.
6 Test Reporting
Generate reports following each test run, distributing these reports as desired via e-mail or other notification (ie, Slack).
Can take and store screenshots.
7 Continuous Integration
Must work as part of a continuous integration system.
Can work remotely, in a device farm, should the need arise to employ this option.
How do I choose a tool to use in testing? I follow this simple mantra:
Problems first, Tools second
I use my knowledge of testing, task analysis skills, modelling and questioning techniques to learn as much about an activity as possible. From there I take the information that I have learned, understand what the problem is and then work out what sort of tool might help.
For example I may have identified through task analysis that I am repeatedly having to set up an environment locally before each exploratory test session. I can then model the system to understand what are the parts that make up the application as well a question my team about how it all hangs together. With that information I can then begin to think about tools that could be used to help setup my local environments. Based on how much time I have and the knowledge me and my team have I could use Docker, Kubernetes, Vagrant, VMs, etc. or I might choose to do something simple like create a Shell script or save a bunch of commands in a sticky note to paste in later.
Ultimately, I know WHY I need a tool way before I know WHICH TOOL I WANT. So remember, Problems first, Tools second
