What kind of features should a test automation tool have?

Hi Guys,
I think 2021 will be an interesting year in our industry and test automation will become much more popular, I am intersting your opinion in this question: what kind of features should a test automation tool have ?

2 Likes

Hi @pawelm,

What features that are relevant depends on the specific use case: what is being tested and who is to be using the tool/framework. Here are some high level feature that could be relevant:

  • Integrations to test case management, bug trackers and event handlers
  • Integrations to pipeline and build management tools
  • handling mobile
  • handling things to web (desktop, mainframe, citrix, legacy)
  • option to test across the desktop not only inside web
  • ease of use, visual programming, low-code
  • options for reusable components, high coupling, low cohesion
3 Likes

Please find below the list of features that a test automation tool should have :

1. Ability to insert code:

No code-less tool can incorporate everything possible. So test authors should have the ability to let testers insert code within the tests for those edge cases. For extra points, don’t limit this to UI-specific code, such as JavaScript. Many teams of automation testing company look at models such as the Test Automation Pyramid and exercising certain actions as web services instead of UI actions. This will allow for faster, more reliable test scripts.

2. Easy assertions:

Recording a scenario is fairly easy to do. As the user takes action on an application, the tool records those actions. However, when a user validates the actions, that does not usually include a web action. It’s something that you do with your eyes.

Because of this, adding assertions within code-less automation tools has always been a pain, requiring manual entry by the user to describe what needed to be validated. This is neither intuitive nor easily expressed.

The process of adding assertions should be as simple as adding the navigational steps. Great consideration should be given to this feature since it’s the most important part of the test script.

3. Reporting:

Adequate reporting is a must-have for test automation, especially when testers of automation testing companies are executing hundreds—or thousands—of tests. Triaging any failures should not require reruns or extensive debugging. An automatically generated report with details on what occurred, screenshots, and even a video of the recording would be very helpful.

4. Conditional waiting:

Record-and-playback tools that require script authors to insert pauses between steps not only are tedious to use but promote a test automation anti-pattern as well.

A more modern approach, called conditional waiting, has proved quite reliable. In this approach, scripts don’t blindly wait x number of seconds before continuing to the next step; instead, they wait until a condition is true and then proceed as soon as possible.

This drastically cuts down on the execution time of the automation suite while also preventing flaky tests. Code-less tools should, at the very least, allow script authors to insert conditional waiting, but it would be even better if the tools took care of this behind the scenes.

5. Continuous integration:

Older record-and-playback tools required testers of automation testing company to click a button within the tool to execute their tests.

In the era of DevOps, test automation is a key ingredient in continuous integration and continuous deployment. Tests should integrate with such pipelines and automatically execute when triggered. For faster execution times, tests should be capable of running in parallel as well.

6. Reusable steps:

Some steps exist in multiple scenarios. For example, logging into an application may be something that the majority of tests must do. Recording this action in each of those tests is what leads to maintenance nightmares. If anything changes with the login step, the test author would need to update every test that contains this.

Code-less tools should allow authors to record common steps that they can then insert into any test flow. This would not only save time in authoring but also promote re-usability and make maintenance much simpler. Similarly, the tools should support the use of variables within tests to allow for the reuse of the state between the steps.

7. Cross-browser support:

Many record-and-playback tools come in the form of a browser extension, which means they may be tied to a specific browser. But many product teams must run tests across multiple browsers. Test authors should be able to record a scenario once on a given browser and be able to play that recording on any other major browser.

8. Control structures:

One of the major limitations of older record-and-playback tools was the absence of control structures, such as loops and conditional clauses. A script author who needed to repeat an action 10 times would need to record that action 10 times and maintain each one of the actions individually.

Also, if there were multiple viable alternatives within a step and the test author wanted to take one action if a certain condition was true, and a different action if it was not, there was no support for such logic.

In coded automation frameworks, testers of automation testing company use loops and if-else clauses to control the flow of scripts. This is a requirement for code-less solutions if the project is to be scalable.

9. Smart element locators:

One of the biggest problems with test automation is that testers of automation testing company develop against an application at a given moment in time. When the application changes, those automation scripts may break.

There’s an opportunity for code-less solutions to have an advantage over coded frameworks by gathering more than one element locator during the recording. If a locator is no longer found during playback, the alternative locators for that element can be tried. This gives the tester of automation testing company a far better chance to develop stable scripts that don’t require as much maintenance.

10. Modification without redo:

Most older record-and-playback tools did not allow authors to easily edit recorded scenarios. If the flow of a scenario changed and you needed to insert a new action in the middle of the scenario, you had to re-record the entire scenario. That’s not ideal.

As the application under test changes, the test scripts need to change as well. This may require adding or removing steps from a flow, or slightly changing existing ones. Code-less automation tools should recognize this as inevitable and make it as simple as possible to accomplish this.

Tempted to start a new thread really, because it’s not that complicated.

  1. No silent failures!
  2. Porting layer supporting remote code execution in parallel/async to any platform
  3. Job or task timeout detection

All things that let you test faster, sooner and with confidence.

1 Like

Here the list of features that test automation tool should have:

  • Dynamic element locators
  • Conditional checks
  • Control structures
  • Easy assertions
  • Easy modifications without redo
  • Reusable steps
  • Cross-browser support
  • Reporting
  • Ability to insert code
  • Continuous integration

Shouldn’t the question be “What kind of features do you require?” A test automation tool/framework can have many fancy features but if you don’t requrie them/will use them what’s the point? :slight_smile:

Example: You can have a UI & API test tool with millions of browser versions supported.
But if you are only going to write API tests, it doesn’t matter and you might want a different framework with less features but easier to use?

3 Likes

I have a few tools in my garage, they are all tools in the fullest sense of the word. When frameworks became a simple re-invention of the “multitool”, some people got slightly distracted. The best frameworks are the ones that are domain-agnostic…but

I have one huge complaint on the topic of high level test execution frameworks. Almost every single one of them is single threaded, or has poor support for multi-role (can control multiple devices/actors simultaneously) test execution in a simple to build and debug language. The JS frameworks are the notable exception to this complex problem-domain which is a very common source of defects. Anyway, that’s my rant on tools for today.

Hi Pawelm,
I recently spent some time studying test automation and I could find the below list of features that a test automation tool should have. Hope this is beneficial to you as well.
1.Dynamic Element Locators
2.Conditional Waiting
3.Control Structures
4.Easy Assertions
5.Modifications without Redo
6.Reusable Steps
7.Continuous Integration
8.Ability to Insert Code
9.Reporting
10.Cross Browser Support

It depend what you need.
Just ensure:

  1. re-use of code
  2. east to understand , if someone new joined in team
  3. Reporting: Plays a very important role, while looking to report one can judge its application or automation failure

1: The ability to programmatically interact with the application reliably.
2: Comprehensive logging so that I don’t need to keep track of what I’ve done.

Also, keep in mind that automation tools are heuristics. The value of using a tool over another depends on its context and there’s no such thing as “the best tool”. There’s only “a good tool for this particular task”.

1 Like

Hello,

I think it completely depends on what kind of features and functionality are you looking for in your project to invest in.

So first I would like to suggest listing out the things that you want in your automation testing tool and then exploring tools as per your needs.

But here are a few things you should consider such as;

  1. Scripting support: The ability to write, execute and maintain test scripts in various programming languages such as Python, Java, etc.

  2. Object Repository: A centralized place to store objects and elements for all the scripts.

  3. Record and Playback: A feature that allows users to record and replay test steps.

  4. Test Reports and Dashboards: Ability to generate reports and dashboards with detailed test results and analytics.

  5. Cross-Browser Compatibility: Support for testing across different browsers, platforms, and devices.

  6. Integration with Continuous Integration/Continuous Deployment (CI/CD) tools: Ability to integrate with popular CI/CD tools like Jenkins, Travis, etc.

  7. Parallel Test Execution: The ability to run tests simultaneously on multiple machines, reducing the overall execution time.

  8. Data-Driven Testing: Ability to feed test data from external sources, such as Excel sheets or databases, to perform data-driven testing.

  9. Debugging and Error Handling: Built-in debugging tools to help troubleshoot errors, and error handling mechanisms to handle exceptions and errors in the scripts.

  10. Third-Party Integrations: Support for integration with other tools such as bug tracking systems, test management tools, etc.

1 Like

Ease of use is the important feature; a test automation tool must have. No matter the technical topic, the finest solutions are always the most direct and elegant ones. And test automation frameworks are no different. A good framework should encourage user’s requirements as simply as possible.

According to me, the test automation frameworks should function like plug-and-play devices. QA team members should be able to use framework even if they don’t properly know how it works.

There should be no learning medium relied on their automation experience or technical skills. It needs to be very user-friendly that inserting, writing, running, and compiling a test script should be seamless without any need of jumping into the underlying mechanics.

1 Like