What piece of automation would you like but haven't found yet?

There’s so much out there now with tools and possibilities of what can be automated.

What piece of automation would you like but haven’t found yet?

2 Likes

A lightweight log parsing suite.

I have worked with application logs for so long, and seen all kinds of tooling, but never had anyone go as far as “simple-logging-for-testers-as-a-service”. By this I mean a tool that accepts logs as an api, and filters then mines them for interesting events. Much like a splunk/ system, but lightweight enough to run portably and at zero cost.

  • I have once used an in-house logging tool that filtered app logs and “colorised” them much like you can do with excel cell colour rules, and supported discrete filters, but only 1 filter at a time.
  • Have also used a tool that ate logs and then filtered them using splunk, much more useful for comparing logs and ML/AI applications

Basically imagine a tool that can do both, work from the commandline/api so you can get live results and use the tool at runtime inside of a test, and also has a GUI. Perhaps this tool would need to be created using a browser based FW like Vue?

5 Likes

There isn’t really anything that I’m really “missing” right now but not sure if this counts as a real answer but I couldn’t think of anything else.

Test automation in VR.

Something like “Primitive” => browsing git repo’s in VR:

I guess you would select locators and write asserts like this? :stuck_out_tongue: I’m not sure how it would work but yeaaaaa

#IKeepDreaming

1 Like

I just love it when people are showing off a tool, which they built like the VR browser, because it’s not really a tool you will use very often. And missing that really day to day utility functionality that would make it something saleable and get those big missing features like the way the tool here does not parse the doc-strings at all, and fails to use colour very well in the demo.

But yes, would love to find a decent more interactive chrome-browser plugin for CSS and xpath locators, (CSS locators are pointless for things like automating lists and automating nested and relative elements) and every tool I’ve tried has either been confusing to use, or has required me write code and wait for things to run in order to verify which strategy makes most sense.

2 Likes

Baretail might be useful. It does not accept logs as an api, but it can filter multiple files real-time.
https://baremetalsoft.com/baretail/

3 Likes

This one’s interesting. In general, I tend to parse the logs directly as for tests, the amount of logs is small enough that parsing the plain text output is fine.

For the larger scale use cases, or when plain text logs are not easily accessible, I tend to use the API to whatever the logging solution is; currently this means writing direct calls to the Elastic API.

2 Likes

I’m like you, have code in my tests, which at runtime we “stream” or hook raw text files into the test framework and then wait for specific substrings to appear. It’s a classic app test practise, and means we have loads of logs, that we could look at, but until they get dumped into something that can be queried using something like Elastic search, and at low cost, finding other interesting strings in those logs just never is easy to do. Piping into a DB where you can do an Elastic search is possibly safer if it bypasses log file wrapping concerns too. I am thinking a log4Net TCP logger on every O/S I guess.

I want a simple web testing automation language (emphasis on simple) that operates at the level of button presses and field inputs so that a newbie tester can follow it, and so can an interpreter using Selenium etc.

1 Like

There is a new version of Selenium IDE available. This tool has a record and playback function. A newbie tester can interpret and modify the code.

There is a course on Test Automation University.

I just noticed, that there is only a Firefox add on available.

2 Likes

Thanks for the heads-up on the new Selenium IDE. It’s not what I’m after, but it’s certainly good to see they’re still developing it.

These days OTT platform is booming. Having said that everyone in the world want to view the content on there own devices irrespective of old / new hardware, old / new browser and with wide range of platforms available in market like Roku, mibox, mobile (Android , iOS) , Browsers (FF, Chrome, IE, Edge, Safari etc) , Smart TV’s etc.

Now the real challenge started for QA to automate all these and to check the consistency of videos with different QoS , Close Caption, Audio / Video sync, Asset transition etc

There are some limited paid tool available in market but they have limitation and can’t support wide range of hardware and costly too.

I really look forward for some tool that can fulfill OTT automation

1 Like

Media center in a box or “OTT” is by nature about being disruptive and providing interactive services in competitive environments, so I would not expect targeted tooling to emerge quickly. That said, a lot of non-functional testing is possible in virtualised or mocked up environments and a lot more hardware that you can attach to and intercept user interaction with is becoming available. I guess it’s always going to be a costly market to test for because of the fragmentation OTT. Not helped by the way video is hard to test without building a few basic things like the “lightpen” and pixel tracker jigs.

I would like to see something that could learn from web logs about typical and atypical user behaviors as well as identify errors, apply that learning to non-production environments for both desktop and mobile, and report on differences and curiosities daily.

2 Likes