I wonder if anyone here has any experience with automation testing for a log service, or any other low level services which might help giving me some ideas.
Looking forward learning from you all
1 Like
Do, you mean “low-level” as in like Syslog, or log-4-net low level? Or are we talking about Elastic logging and search type logging?
One of the main issues I’ve had with log testing is lag between the event and the log appearing. Particularly Microsoft Azure, you can often wait several minutes between something being done and the corresponding log event appearing. Azure is probably the worst I’ve experienced, but I would expect any automation tool to outperform a logging service. You just need some backoff/retry mechanism to cope with the delay. And obviously if you’re checking for many events your overall execution time could become quite excessive.
Having said that, actually looking for a particular event and checking for its content is pretty trivial. I’ve often used date/time bound URL’s then searched for particular keywords or references in the response. For example if the log messages are bound by a session ID, you can query the session ID then check each event has a corresponding log message.
I had some automation tests in an old project that involved checking logs for errors. It was very low level logging. We moved logs with errors away to not get fails due to old test results. There were never any issues with the log entries not being there after the action happened. Our tests helped highlight that nearly every log had a different syntax and helped get them more similar.
You need to elaborate about it a bit in order to get better insights.
Are you automating a product where the verification is in the log files OR you are checking a log system iteslf. There are very different.
Do a step back, you should have a set of functional requirements, acceptance criteria and performance requirements and for which purpose do you need it.