I checked the Alchemy Testing web site. Looks like it is essentially a recorder on top of selenium. There’s the Selenium IDE available as well, so I wonder why can’t we just collaborate and build a worthwhile recorder that would not require comparing a gazillion different ones.
There’s some pretty promising record + AI structure things out there, but I wonder how many different ways we need to build similar blocks and platformize them into new product names.
As a member of Selenium Project Leadership Committee, I tend to be pretty proud of how much business and tools people build on top of Selenium. But it does not feel like we, in community at large, are helping our users understand things. We provide them new tool names, similar looking arguments, and sometimes like with a demo I watched today, make sales claims of ease comparing to things that weren’t true for Selenium for 1,5 years on what it takes to set up browser versions and drivers to run tests.
It’s not Alchemy that I am picking here. We have one of those built where I work too. I just can’t help but thinking that there is a bigger problem with discoverability and we aren’t making it easier with the options.
I appreciate your insight and experience. I would invite you to look a little deeper at Alchemy, however. It is well beyond a recorder on top of Selenium, although it has that as the basic feature.
I think the additional features of Alchemy make it a very worthwhile tool. It is designed with every level of tester in mind. From someone just starting out that has no testing or coding experience to someone that has a great deal of experience in testing or coding.
As I mentioned, I have used UFT (over 20 years) and TOSCA in the past. I would choose Alchemy over either of them (for web application testing) for various reasons.
A big one as you addressed earlier is budget. You said,
“Operating a low-code tool also requires budget. Usually two kinds of budget in my experience: money out of pocket for a commercial tool license AND time to learn and use the tool.”
Alchemy solves that problem. It is free. The learning curve is minutes. And if you need to do something beyond the record and playback, the team has videos on YouTube that address the commonly used functionality (coding a custom action, data driving, etc) that you can learn generally in less than 15 minutes.
I would love it if you would download it and check it out. I think you would be quite surprised at it’s capabilities. I would love to hear your feedback about it. Feel free to reach out.
Since I watched the video that was over a minute, the learning curve is not minutes. It is most likely reasonable, just like with many of these tools.
I recognize I simplify calling it a recorder. I see there are other supporting features. Four low code tools in a day seem to make me simplify what I am watching. Not paying thousands a month per user out of pocket is most definitely a benefit.
I also had a colleague in between projects and used them as a guinea pig on high code version this week, and observing the work breakdown for claims of low code tools saving time.
One blob of time went to maintenance that a more protected / packaged test development environment could avoid - python version and library dependencies, world changed and error messages on figuring things out aren’t quick and routine, but doable. This is something where low code would either not have it or if it had, the time would be more than blob on low code as the error message would be big and scary.
Second blob of time went on authentication, or essentially learning about how to save up authentication so it does not have to be repeated. The recipe is easy when you know it, but when you don’t, it takes expecting that one exists (more likely with high code) and learning by reading. I am sure similar credential saving mechanisms are packaged on high and low code, but in an environment where people don’t feel in control such as low code, the blobs of time wasted would multiple before finding the way around the 2FA.
Third blob of time went on the first test case, which means essentially to the fact that the tester at hand needs to learn to operate the target system AND the target system being a demo system is low of data and thus has a lot of features crippled. You can get to states where you can’t go back nor forward, and it takes a bit of twisting your head to understand the role of data and how you control that. Observing says they chose things possible rather than things important to test to show progress. This would be different in low code with business user, they would either be totally blocked or know enough of the application to use it manually to start from.
Conclusion: 3 blobs, potential to saving time and getting stuck worse with low code seem balanced but more likely to save time with one blob (environment).
Has anyone seen good breakdowns of categories time goes to when writing automation?
If a product is free I’d want to check how they sustain a development team. Either it’s freemium and they have other features that make them money (Cypress) or it’s an open source project backed by donations from big corps (Selenium) or it’s backed by a large company like Microsoft for some commercial reason that is yet to be obvious (Playwright).
What is the risk of your company moving to a platform and suddenly finding that fully featured licenses are becoming paid-for from date x? Katalon took that approach a few years back.
This is a really useful point - I think you’re saying that with a low-code tool, you make it more accessible to business users, business analysts or testers with good knowledge of the system? I’m assuming that people very familiar with high-code solutions, such as SDETs, usually lack knowledge somewhere else (perhaps domain/business knowledge or testing skills). Which creates a risk of automating the wrong thing. There’s always a trade-off but writing really good code should be secondary to writing the right code I guess!
For us, all of Alchemy’s current features (which are quite a lot) will stay free.
Current revenue is generated by premium support (above and beyond our Knowledge Base and Community) and Alchemy Gridworks Cloud credits, which are very low cost to the user.
There will be various enterprise level features in the future, for example enhanced execution analysis, or organization level sharing of assets, those will be paid add-ons. But the current, very full featured product will remain free.
Feel free to ping me directly if you have questions or would like more info.
That is a very good point, @danuk. Several of my past engagements involved systems and industries I had no prior knowledge about and had to rely on business analysts/SMEs to write good requirements and also consult with them directly, which of course added time to get the job done. With low/no code solutions, the BA/SME could have created the framework of the test themselves quickly and handed it off later to the SDETs to add the more robust and dynamic code it may need (like making it data driven, etc) before incorporating into an execution suite.
This is a problem I have been dubbing with the concept of contemporary exploratory testing. I have been continuously hoping for a tester who can both test and code, and while some exist, a lot is either or. I am regularly reflecting if asking for both is unreasonable, and faced with the reality that a lot of manual testers (with test cases) can’t test well enough to provide the level of results needed from testing, especially when domain and target app is not their home ground.