Great questions raghu - see my responses below:
1 - Does it allow us to code in case we need custom functionality also?
If yes, then which programming languages does it support. If no, then is it in your roadmap?
No, and adding code support is not on the roadmap. Our goal is to enable users to automate as many scenarios as possible without writing code. The closest thing you’ll have to writing code in our system is Variables, which is akin to Excel macros. Variables allow you to set up more complex assertions, generate random values, things like that. More info on that feature here: Variables | Reflect
2 - Does it allow us to debug tests, i.e. run them step by step?
Are there any limitations in your debugger like needs too much memory, can be used only for certain parts of the test cases and not the entire test case etc.
We have support for re-running tests in our recorder up to an arbitrary point. Adding step-by-step debugging is on the roadmap!
Can we inspect any variables if we need to during debugging?
Yes we provide diagnostic information for variables but it’s more simplified vs. what you’d have if you were debugging a Selenium test, for instance.
3 - Code or no code, does it have refactoring features?
E.g. Does it allow you to extract a test or its steps and convert them into some kind of reusable utility or test?
Yes, we support test re-use via our Test Composition feature (Test Composition | Reflect) and refactoring tests via our “re-record” feature (Editing / Re-Recording | Reflect)
4 - Does it allow certain tests/“functions” to run before a group of tests just like TestNg?
E.g. Do login before every purchase related test. But, don’t login for product searching tests.
Yes you can do this via our Test Composition feature (e.g. every test that relies on login shares the same login test, but the actual execution of the login test runs in a separate browser context for each run so no state is shared across tests). More complex test orchestration is on our roadmap!
5 - What is the format in which you store test cases behind the scenes?
Eg. Java tests would be stored as Java files and class files. So, does it store tests in some proprietary format or common one like json, xml etc. ?
We store it in a proprietary format and expose test data via our API. We’re planning on adding integrations to test case management tools and bug trackers in the near future so tests data can be synced with these systems.
6 - How does it handle versioning and test case reviews?
Can we use Git & Github with it?
Tests are versioned within our system, but we don’t sync with Git. To be honest I’ve never seen syncing from a third party system to Git done well. It either is a one-way sync which doesn’t seem terribly valuable or it’s a two-way sync that runs the risk of overriding data if you’re making changes both via Git and via the web interface. Have you seen a product that does Git sync well?
7 - Is every or at least major feature well documented?
Does every feature have a link to the documentation for that feature?
Our documentation is pretty comprehensive! https://reflect.run/docs
8 - How do I migrate away from your tool when I need a lot of customization?
Is there any tool which automatically converts the tests into documents or code?
No, although I’d argue that any tool you use has an inherent lock-in cost. For example there’s no easy way to migrate code between open source frameworks. Certainly some vendors provide an export to Selenium though.
9 - Which learning resources do you have for beginners to your tool?
What is the cost of these resources? Do any of these resources cover realistic use cases or just toy examples?
Our docs cover all of the features of our tool, are completely free, and we also have videos embedded in the docs that cover the most important features. The videos show tests against both demo sites and real sites, but all of the sites are publicly accessible so that you can follow along step by step. Reflect also has a free tier so you can sign up and try it out.
10 - Do you have any publicly accessible forums where people can ask questions about the tool?
Personally, I’d prefer forums such as stack overflow which are not owned or run by the tool company.
We get questions about our tool periodically on public forums. Here’s an example thread on Reddit: https://www.reddit.com/r/QualityAssurance/comments/l287cu/which_noncoding_automation_tool_would_you/
11 - Do you have any publicly accessible forum where people can suggest features and bugs, and also see their status?
No but we’re very open to feedback - you can always email us at support@reflect.run!
12 - How is this tool better than its competitors?
We’re very focused on test accuracy and maintainability, and a lot of the design decisions about the product and built around those two concepts.