Everyone has a favourite log inspecting tool, mine used to be excel, but. Some people will tall you to use elasticsearch and I’m just low on time sometimes to learn a new tool. Or to pay for it for that matter. (I found some inspiration for my angel of attack in an old 30 days of testing topic : What's the trickiest bug you have caught using logs?)
Excel does not deal with huge huge logs that well, it will let you colour in all records that contain certain words for example, and if you have the time you can even automate in VBA/macros, but it’s not performant enough for everyone, so it’s out of the window.
Notepad++ , great, but it really only has search and highlight, it does not filter and remove things you don’t want to see at all. It’s also destructive, you can delete noise easily, but it then gets hard to get that back.
Grep, is great, search helps, but it lacks context as a tool, because it’s just a bunch of greps or filters and joining them up , and remembering them gets tricky.
Has anyone found anything open source-like that lets you parse text or even xml logfiles? It’s probably viable to convert from text/csv to xml, for most of our uses. So right now I’m tempted to write a tool that joins up and concatenates logfiles and then lets me filter. Anyone done this before? Lessons?
3 Likes
I remember using this or something like it: LogViewer - Home page which will colour code your log entries based on RegEx. It can tail a log and immediately give feedback during testing to show, for example, errors or particular behaviours like DB in or out.
I can’t imagine it’s difficult to find a set of libraries in your favourite scripting language and shove them together, honestly. I used to do this with Ruby and then Python to make log viewers and file differs for bespoke reasons. Whatever gets you the observability of your dreams.
I find that I always need a tool that gives immediate feedback during testing so that I can shorten the distance between observation and inference while I explore, so I’d always go in that direction whenever possible.
1 Like
That LogViewer is a bit limited compared to what is on github now. So far I found a decent github project, which is marked as free for private and commercial - (it has not got a specific repo license chosen.) called Advanced Log Viewer (C# code) GitHub - Scarfsail/AdvancedLogViewer: Advanced Log Viewer . The code is not too crazy complex, so I might adopt it.
But yeah @kinofrost I like the “filtering” using code approach of using a scripted tool which is just so powerful when you can go ahead and add your own custom searches and even do some data extraction and exporting if you want to in a script like Python. Or even Ruby, although I’ve never learned any Ruby myself, it sounds like using a language you are an ace at already means it’s always available.
1 Like