Automated Exploratory Testing

(Rosie) #1

I came across this, it may be of interest/use. Anyone have experience with it?

AET (acronym formed from Automated Exploratory Testing) is a system that detects changes on web sites. AET is designed as a flexible system that can be adapted and tailored to the regression requirements of a given project. The tool has been developed to aid front end client side layout regression testing of websites or portfolios. In essence assessing the impact or change of a website from one snapshot to the next.

(Hannah) #2

Thanks for sharing! Had a quick look and seems like an interesting tool. I might have to give it a try. I like that it does JS logging, status codes and cookie testing.

Shame though that it’s been plugged as an “exploratory” tool. Automated exploratory testing seems like an oxymoron to me.

(Mark) #3

Any tool that claims to automate exploratory testing turns me sceptical right off the bat (this isn’t the first one I’ve seen that claims to have automated the un-automatable). Especially as it appears to be a front end heavy tool (the only mention of API is status codes). It seems like it has some value in a monitoring capacity, giving you feedback on what has changed between states/builds/releases. However, I would be very much aware that when using a tool like I would use it as a means to stem actual exploratory testing based on results.

Also running on Vagrant / Virtualbox means you are hampering your feedback loop which is one of the key benefits for automation.

Disclaimer I’ve not used this tool

(Garry) #4

Thanks for sharing the link @rosie Seems like an interesting tool but as others mentioned calling it an “Automated Exploratory Testing” is bit skeptical. As far as I understand, it is more of performing the impact analysis from build to build. Moreover, as it states it is to aid front end client side layout changes so I wonder how it picks up the business process changes in the middle layer but surely worth to have a look at!

(John) #5

I do have to agree with the others about being skeptical. Exploratory Testing is a very cognitive-driven approach to testing. You don’t know what you’re going to do after your initial tests - you don’t know what you’ll see or what you may think of next, based on the results of your initial tests.

I also feel the software development industry is overly glamorizing test automation. So I’m getting the vibe that if you want to be considered leading edge or high tech, you must use the word “automated.”

I will say that understanding the impact of a software change would be very powerful and extremely useful for testers. I tried to get the Dev Manager to use some Microsoft tools that would show the different modules impacted, but it was to no avail.

(Paul) #6

I agree that calling it an “automated exploratory” testing tool is dubious going on mendacious, however it does seem to be a very useful web page style and content checker. It would certainly be useful in regression.

Another thing to note is that it does not have many contributors (12 I think) and seems to be tied to a small digital consultancy in London called Cognifide. That’s probably alright but the defect list on github is already quite long, dating from 2015, which suggests that supporting this in the future isn’t exactly their greatest priority.

(Dan) #7

Hmm… I share the scepticism here, but I’m going to play devils advocate for a second to spark some thoughts…

The link in the OP from Rosie talks about “regression testing”.
Now we all know that we can utilise a scripted approach (common) or an exploratory approach (probably less common) for regression testing. The scripted approach relies on having those expectations to “check”, whereas the exploratory approach investigates the risks and likely areas where regression problems may occur.

If this tool can, as it claims, highlight regression problems where there were no real previous expectations at a level related to individual “checks”, then does this mean the tool actually does relate closer to an exploratory approach rather than a scripted one?

Now, obviously, its not conducting the exploratory testing activities automatically on its own… That’s impossible, but perhaps this tool is aimed at purely supporting ET rather than scripted testing? (even if the description of the tool is badly written)

There’s my thought grenade chucked in the ring :grin:
What do you think?

(Paul) #8

Not sure I agree @danashby. By documenting every step you do in an exploratory test, say of a web application, after you think of it but just before you execute it (like in some SBTM) you can treat your exploratory test as the sum of many just-in-time “scripted” tests.

If, let’s say, instead of manually executing each “scripted” test in the above you wrote an automated test in Selenium or UFT to do it, you would be using a test automation framework as part of your Exploratory Test. That still does not make Selenium particularly useful for exploratory testing nor does that distinguish it from using any other tool in the same way.

Similarly, the action of AET would probably yeild the same results if you used it in the above way or as part of a set of scripted manual test cases. I am not at all convinced that this means it can be called an automated exploratory testing tool.