Tools to Aid exploratory testing sessions

Good morning,

Thinking about tools that can aid exploratory testing /product reviews can anyone add any thoughts here.

Basically I’d be looking for a video capture tool which can do lower resolution (so we’re not producing gigs of footage when recording 1-2hr sessions).

I’d like to be able to add bookmarks when there are points of interest (so someone watching can jump to the bookmarked areas).

Possibly creates captions for the voice overs.

The use cases would be either capturing a exploratory session to then review after (so you don’t lose flow by writing notes etc), or to give critique of features and then pass back to the PO and team to review.

All this might be standard for video capturing (I’m not a twitch or streamers etc :wink:) but looking for any advice. Using OBS studio at the moment but not really useful beyond the capture.

Thanks

1 Like

XRay Exploratory Tool

Yattie - Systematic Exploratory Testing With YATTIE | Ministry of Testing

All the best!

3 Likes

It sounds like what you’re looking for is quite far outside the scope of a tool that’s designed for aiding exploratory testing specifically. You might have more luck looking at video editing software in general. As for basic capture, Awesome Screenshot has served me well for several years.

2 Likes

Videos are awesome and all, but I’ll challenge you to think about how sharable and discoverable the information is within them is.

Unlike with a document you can’t just scan to see where things are or search/find for information (you’d have to know to jump to the bookmarks and then watch them all). Nor can you make it visually there in one page like in a mind map for example.

  • Could your bookmarking idea be replaced with a screenshot and a short commentary?
  • Would a critique be taken better and more tailored via an in-person debrief rather than a demo?

What challenge are you facing that you’re trying to solve with video?

1 Like

Bug Capture (formerly Bird Eats Bug) Is a great tool that may be helpful in what you are looking for. We use it for evidence capture and debugging customer issues.

1 Like

For video clips for communication, such as showing someone on a digital bug report what you’re trying to do or showing the timing to reproduce something I’m a fan of Cockos Incorporated | LICEcap.

I’ve tried using video for exploratory work. I did like that I could review my work to see if I actually did what I thought I did, and it helped me to catch intermittent problems or fleeting screen artefacts. I sometimes reviewed what I did to see how I might improve on it.

However that came with some important drawbacks. I think explaining them might help your search for tooling (with regards to your session review use case)

Because the exploration inherent to testing doesn’t expose my internal structure (what my hypotheses may be, risks I’m testing for, setup I’m doing, branching I’m doing, tangents I’m going off on, things I’m noticing, emotions I’m responding to, techniques I’m applying, when I defocus, etc) or external structure not on the screen (other devices, conversations with others, printed materials, windows I’m not recording right now) I have to expose that in my notes.

I have to balance the effort I put into these notes. I want the notes so that I can build the story of my testing and update catalogues and artefacts, and for some proof of my efforts, and to combat my short memory. However, every second I spend writing notes is a second that I’m not exploring the product, and stopping to write notes can interrupt the flow of an idea.

Video feels like a good way to lessen the pressure on note-taking, because your activities are recorded. But they’re not really, because the video doesn’t contain your internal structure. It is not capturing your testing, it’s capturing what your testing looks like to someone standing behind you. If you want the video to make sense to you later, or to anyone else, you have to write notes either in view of the recording or with timestamps and the like, or it’ll simply fail to show what you were trying to do and what you learned by doing it. You end up writing notes for the imaginary viewer of the video. This may actually mean more notes, or a different style of notes, which may impact your testing.

Another issue with video is how off the rails your exploration goes. If you start a session and you’re on-mission and suddenly you find a big problem, or you start coming up against bugs, you then have to stop your recording and name your file and so on. Or you have to spend time investigating the bug and writing a report. This effort of starting/stopping/pausing/saving/naming/updating artefacts with your recording can give your testing more inertia, and make it feel less flexible. It feels like a little emotional cost every time you want to stop or start or change.

I think video works at its best when it’s not for communication or archive, but as a way to expose intermittent problems, provide review for recent activity, or even as a way to improve your own attention - you can concentrate on multiple things by watching everything multiple times. I sometimes used to record folder watchers and filtered log tailors and such as I test, because it’s easier to see logs in real time than to match up the timestamps with what you thought you were doing when they appeared. One advantage of this sort of thing is that you don’t have to store these videos, unless they contain something you want to save with comments, or use in communication surrounded by explanation. You can make your internal ideas about mission and risk and technique explicit when it’s useful to do so.

I think video does have a place, especially for particular cases, and if you can do all the recording for free and have inexpensive and secure storage for it, then the cost can be relatively low. Don’t forget that you have to pause the video for lunch, or create multiple videos of one session. And that screen recording takes system resources. I’d definitely try it with any old recording software first and see how your idea of it mixes with reality, as that was illuminating for me.

Another possibility is voice recording. You may find it easier to speak what you’re doing than write it. You can then use STT to convert to text, if you like. Personally I need to write to think properly, and I rarely worked in environments that would love me loudly saying what I’m doing.

The vast majority of screen recorders only seem to record screens. Concerning the extras you’re looking at I think that Yattie is moving in a positive direction to handle these sorts of problems, and it feels like it’s written by someone who wants to integrate video evidence more seamlessly into the note-taking process in a way that’s compatible with session-based work. It’s made some good moves towards being a professionally usable tool, too. It’s worth considering for any tool what happens if it crashes - do you lose all your valuable progress?

Incidentally Windows has a game bar feature with a screen recorder on it. Select the window to record as the active window, press Win-G, click the camera at the top. A little recording window appears. Click the small circle icon to start recording, then click anywhere on the screen to return to what you’re doing. Press Win-G again to be able to stop the recording. Start/Stop recording has a shortcut, too: Win-Alt-R. Certainly has its limitations, but a useful thing to know.

1 Like

Hi,

The problem I am trying to solve is a few.

Focus - so when reviewing a product, e.g. from a usability/UX perspective its good to be able comment as you are reviewing without having to break away.

In person reviews might be difficult to do (distractions and interference as well) so the video allows an uninterupted session and the person recording to speak what is on their mind at the time. This video of course may then be used to seed a discussion/in person review.

I can do this end of day when the office is quieter and /or less distractions.

What I’m doing principally here is looking at options for people to utilise, the video idea may or may not work, but something I am initially keen explore a bit more.

Thanks for taking the time on the considered post. Very helpful.

Unfortunately we’re still on W10 so don’t have the built in recorder.

A thing that has recently spurred me on a bit to look at a video option was following doing some review with the rest of the team on a product recently. As a fresh pair of eyes on the product I was highlighting “questions” (not issues, but questions, or things that weren’t clear to me that may or may not be issues) and it just got me thinking that, as someone who’s busy in work, there may be benefits to me taking time to do some recordings and then passing to the team to review. However, if I just given them a video then they need to sit there and watch it all (hence the bookmarking thing). But I also think that perhaps building in perhaps weekly sessions to go over features all together is something that might be good to do anyway.
I’m not a video editor, or streamer etc so really don’t have much knowledge of the various video editing apps, so perhaps (as someone mentioned above) this might be something to explore more (rather than per se, exploring tools marketed at ET).

The recording option to me, felt like an option for sitting there uninterupted and letting me say what is on my mind at the time.

Hi, Testuff test management has a built-in video recorder, with which you can record your testing (and bugs) while running the test. The video is automatically saved with the execution, can be sent to a tracker (or not) and more.

Disclaimer: I work for Testuff.

Have you tried pairing?

Having one tester drive and the other tester taking notes as you go? That could be a cheaper / lower tech solution to investigate :smiley:

I like this concept and use occasionally as this also gives others an insight into how your brain works. However for some occasions, it might be later in the day, or limited availability of others etc so still an option I want to explore a bit more (apologies, it sounds like I’m making loads of excuses :smiley: )

1 Like

Unfortunately we’re still on W10 so don’t have the built in recorder.

W10 has the recorder, but given that it’s original purpose was to record gaming footage I suppose it’s excluded from the install for a work environment.

there may be benefits to me taking time to do some recordings and then passing to the team to review

I don’t feel like I’ve understood what you’re going for. I’m not sure what a product review is, I think. Is the idea that they review session footage as a kind of session debrief? For every session you do? Or a sort of presentation of product features, like a sales video?

I’m not a video editor, or streamer etc so really don’t have much knowledge of the various video editing apps, so perhaps (as someone mentioned above) this might be something to explore more (rather than per se, exploring tools marketed at ET).

Editing takes longer than one might assume. It may be easier to make multiple short videos for each thing you’re trying to capture. If you’re going to re-use the footage over and over then maybe it’s worth it.

If you’re going ahead and trying it then for professional video editing for free I’d point you at DaVinci Resolve. There’s a reasonable chance that’s too much for what you’re trying to do and there’s a learning curve (but it’s free and you don’t have to actually use most of the features).

If you don’t mind the watermarks and want to just try something without installing anything then I’ve been pointed at the online editor ClipChamp before.

The recording option to me, felt like an option for sitting there uninterupted and letting me say what is on my mind at the time.

That’s what my internal structure essay was getting at - expressing what’s on your mind vs seeing what’s on your screen. It’ll depend on what you’re trying to achieve. Yattie lets you associate notes with video. You could record you taking the notes as you record the screen, so they at least match up in terms of timing. You could include timestamps in the notes that refer to the video. You could edit the video with text overlay. You could record spoken audio with the video. You could write text near the video. Or you can stand next to the video as it plays and talk about it.