New codeless automation tool

Hi everyone!

I’m the co-founder of Reflect and I wanted to share our product with this community. (To the mods: I see the suggestion is to post new products to the ‘Cool Projects’ category but that link does not work so hopefully I’m posting to the right place)

Reflect is a no-code test automation tool for testing web applications. We know that record and playback tools have been around for a long time and have had some major drawbacks, specifically around test brittleness and lack of expressibility. We’re taking a different approach vs. other tools with the goal of making tests easy to create and maintain. While we’ll never let you achieve the level of expressibility of code, we think there’s a lot of value in a tool that hits the 80/20 rule; letting you test the 80% most common scenarios with 20% of the time you’d need spend with other approaches.

We can accurately record a ton of actions - I don’t want to list them all here for fear of this looking too much like an advertisement. But I’d love to get your feedback on what you want to see in a no-code tool and what kind of scenarios you’d need us to support in order for you to get value out of a product like ours.

2 Likes

Here are some questions.

1 - Does it allow us to code in case we need custom functionality also?
If yes, then which programming languages does it support. If no, then is it in your roadmap?

2 - Does it allow us to debug tests, i.e. run them step by step?
Are there any limitations in your debugger like needs too much memory, can be used only for certain parts of the test cases and not the entire test case etc.

Can we inspect any variables if we need to during debugging?

3 - Code or no code, does it have refactoring features?
E.g. Does it allow you to extract a test or its steps and convert them into some kind of reusable utility or test?

4 - Does it allow certain tests/“functions” to run before a group of tests just like TestNg?
E.g. Do login before every purchase related test. But, don’t login for product searching tests.

5 - What is the format in which you store test cases behind the scenes?
Eg. Java tests would be stored as Java files and class files. So, does it store tests in some proprietary format or common one like json, xml etc. ?

6 - How does it handle versioning and test case reviews?
Can we use Git & Github with it?

7 - Is every or at least major feature well documented?
Does every feature have a link to the documentation for that feature?

8 - How do I migrate away from your tool when I need a lot of customization?
Is there any tool which automatically converts the tests into documents or code?

9 - Which learning resources do you have for beginners to your tool?
What is the cost of these resources? Do any of these resources cover realistic use cases or just toy examples?

10 - Do you have any publicly accessible forums where people can ask questions about the tool?
Personally, I’d prefer forums such as stack overflow which are not owned or run by the tool company.

11 - Do you have any publicly accessible forum where people can suggest features and bugs, and also see their status?

12 - How is this tool better than its competitors?

13 - This one is for a user of any tool and not for the tool company.
Do some frequently used features of this tool require too movement with mouse/trackpad?

Too much movement of mouse/trackpad can potentially cause discomfort or even Repetitive Stress Injury/RSI. In contrast, RSI is alleviated, but not prevented or cured, by auto complete in coding IDEs which low code tools don’t have. So, you hope that the UX design of the low code tool does not take too many steps to do things, especially simple things.

PS -
I guess these could be used as a template for any low code, no code or codeless automation tools.
I think I could ask more questions, but I’ll stop for now.

2 Likes

At my company, we are already using Reflect and my colleague wrote her review of the tool. We think you guys are doing great thing! https://abstracta.us/blog/tools/reflect-review-the-new-automated-web-testing-tool/

Great questions raghu - see my responses below:

1 - Does it allow us to code in case we need custom functionality also?
If yes, then which programming languages does it support. If no, then is it in your roadmap?

No, and adding code support is not on the roadmap. Our goal is to enable users to automate as many scenarios as possible without writing code. The closest thing you’ll have to writing code in our system is Variables, which is akin to Excel macros. Variables allow you to set up more complex assertions, generate random values, things like that. More info on that feature here: Variables | Reflect

2 - Does it allow us to debug tests, i.e. run them step by step?
Are there any limitations in your debugger like needs too much memory, can be used only for certain parts of the test cases and not the entire test case etc.

We have support for re-running tests in our recorder up to an arbitrary point. Adding step-by-step debugging is on the roadmap!

Can we inspect any variables if we need to during debugging?

Yes we provide diagnostic information for variables but it’s more simplified vs. what you’d have if you were debugging a Selenium test, for instance.

3 - Code or no code, does it have refactoring features?
E.g. Does it allow you to extract a test or its steps and convert them into some kind of reusable utility or test?

Yes, we support test re-use via our Test Composition feature (Test Composition | Reflect) and refactoring tests via our “re-record” feature (Editing / Re-Recording | Reflect)

4 - Does it allow certain tests/“functions” to run before a group of tests just like TestNg?
E.g. Do login before every purchase related test. But, don’t login for product searching tests.

Yes you can do this via our Test Composition feature (e.g. every test that relies on login shares the same login test, but the actual execution of the login test runs in a separate browser context for each run so no state is shared across tests). More complex test orchestration is on our roadmap!

5 - What is the format in which you store test cases behind the scenes?
Eg. Java tests would be stored as Java files and class files. So, does it store tests in some proprietary format or common one like json, xml etc. ?

We store it in a proprietary format and expose test data via our API. We’re planning on adding integrations to test case management tools and bug trackers in the near future so tests data can be synced with these systems.

6 - How does it handle versioning and test case reviews?
Can we use Git & Github with it?

Tests are versioned within our system, but we don’t sync with Git. To be honest I’ve never seen syncing from a third party system to Git done well. It either is a one-way sync which doesn’t seem terribly valuable or it’s a two-way sync that runs the risk of overriding data if you’re making changes both via Git and via the web interface. Have you seen a product that does Git sync well?

7 - Is every or at least major feature well documented?
Does every feature have a link to the documentation for that feature?

Our documentation is pretty comprehensive! https://reflect.run/docs

8 - How do I migrate away from your tool when I need a lot of customization?
Is there any tool which automatically converts the tests into documents or code?

No, although I’d argue that any tool you use has an inherent lock-in cost. For example there’s no easy way to migrate code between open source frameworks. Certainly some vendors provide an export to Selenium though.

9 - Which learning resources do you have for beginners to your tool?
What is the cost of these resources? Do any of these resources cover realistic use cases or just toy examples?

Our docs cover all of the features of our tool, are completely free, and we also have videos embedded in the docs that cover the most important features. The videos show tests against both demo sites and real sites, but all of the sites are publicly accessible so that you can follow along step by step. Reflect also has a free tier so you can sign up and try it out.

10 - Do you have any publicly accessible forums where people can ask questions about the tool?
Personally, I’d prefer forums such as stack overflow which are not owned or run by the tool company.

We get questions about our tool periodically on public forums. Here’s an example thread on Reddit: https://www.reddit.com/r/QualityAssurance/comments/l287cu/which_noncoding_automation_tool_would_you/

11 - Do you have any publicly accessible forum where people can suggest features and bugs, and also see their status?

No but we’re very open to feedback - you can always email us at support@reflect.run!

12 - How is this tool better than its competitors?

We’re very focused on test accuracy and maintainability, and a lot of the design decisions about the product and built around those two concepts.

Guys, if your Organizations are looking to get a scalable Salesforce test automation then they can find the best solution in codeless test automation. ACCELQ’s Codeless test automation addresses most of the challenges that QA teams face in Salesforce test automation so that companies can get a smooth experience with their Salesforce testing.

Let me tell you what all challenges which need to be addressed:

Adapting to Salesforce Software Updates
Complex Salesforce Web Components
Adjusting to Different Salesforce Versions
Compressed Configuration and Implementation Cycles

Now, lets discuss about “Codeless automation tool i.e: ACCELQ”.

  • ACCELQ is a cloud based, continuous testing platform for functional and API testing needs. accelQ is offered both on SAAS and On-Premise models. ACCELQ is an official Salesforce ISV partner, which ensures that ACCELQ is aligned to Salesforce Releases and that the customer gets smooth Salesforce upgrades with robust automation testing. ACCELQ Automation is a comprehensive test automation solution built on a codeless architecture. ACCELQ adapts the concept of Lifecycle Automation, where automation is fully integrated with the test lifecycle rather than being a technical silo. All aspects of test cycle such as test design, development, execution and analysis are automated.

  • ACCELQ’s AI-driven test automation cloud, optimized specifically for Salesforce Dynamic Technology, makes it the best choice for Salesforce testing customers. ACCELQ Salesforce Content Pack accelerates the Salesforce test automation by three times. All you have to do is write your validations in simple English or drive them directly from the UI. The automation adapts to frequent configuration changes in your Salesforce implementations and seamlessly handles Salesforce specific elements. ACCELQ uses ready-to-use test automation assets with zero technical complexities to help you give complete visibility on the quality of Salesforce implementation. With three times faster test cycles and significantly lower quality cost, you get a 70% reduced test automation effort.

ACCELQ Features:

The following are the features of “ACCELQ-Codeless automation tool”:

  1. Easiest and most sustainable way to automate web and API testing with ACCELQ:

    Automates all aspects of Quality lifecycle from Test Design all the way to Test Planning & execution
    Ensures design-first approach with inbuilt modularity; No need for custom frameworks
    CI ready, scalable and secure for your enterprise.
    Enables manual testers and business analysts to automate without the need for programming
    Enables in-sprint automation with industry first virtualized abstraction for functional testing
    Seamlessly integrates in your eco-system of Tools like Jira, TFS, Jenkins, Bamboo etc.

  2. ACCELQ Automates Test Design Maximizing Coverage & Reusability:

    Universe driven visual test design keeps business focus while generating Application blueprint
    Embedded frameworks bring modularity for faster development & lower maintenance
    Develop test scenarios based on predictive analytics and path analysis
    Model UI and Data flows to maximize coverage with auto test generation

  3. Natural Language Programming:

    Through an innovative natural language approach, ACCELQ delivers complete automation solution without having to write any frameworks or learning a programming syntax. All test assets are accessible to the entire team without any technical barriers.
    Complex test logic can be developed in English abstraction with a browser based logic editor.

  4. Intelligent Element Explorer:

    Powerful & intelligent element capture technology creates robust automation
    Self-healing automation drastically reduces maintenance and accelerates ROI
    No coding needed, no matter how complex and dynamic your application

  5. Extendible & Open source aligned:

    Easily extend to support newer technology and custom controls without programming complexities
    Enable the entire team to automate, its fast and very low maintenance

  6. Automated Test Generation:

    Abstraction concept allows in-sprint automation by providing ability to develop full blown automation test assets while the application is still under development. When your app UI is ready, reconcile with the concrete application. Self healing capabilities ensure reliable test execution in case of unforeseen changes to application UI or environment.

  7. Cross browser cross platform executions on cloud or on-premise labs:

    Dynamic live results views with actionable reports to trigger reruns
    Parallel executions supported for quick turnarounds
    Email notifications and more to fit into your process and enable quick decision making
    Runs can be scheduled, recurring or be triggered by CI workflow