Exploratory Testing Tools

This is a great idea.

In fact, I was already ruminating on a reports improvement/customization Epic and this seems like an excellent place to start.

I’ll ping you here to take a look when I get it added in the next few days!

Thank you for this @simon_tomes!

My first question is “How could we auto-generate a summary?” and this seems like a wonderful place to start.

As I mentioned to @oxygenaddict above, I was already contemplating reports and how they needed some love and improvements. :slight_smile:

I have also reached out to some of the folks who give sessions here on MoT about better reports to pick their brains as well, so hopefully that leads to some great ideas/options for improved reporting.

Additionally, YATTIE is getting another release with a “pre and post flight” checklist feature (which allows you to set checklists for before and after your session and designate them as mandatory or not) and this could tie right in! So I’m imagining a setting that you can flip to enable the summary box on session completion.

Thanks for the feedback!

I’m curious what you were expecting for an exploratory testing tool? All of the exploratory tools I found in my research are pretty prescriptive on the chronological cataloguing of evidence captured, so it seemed like a good place to start before branching off into something more generally useful!

That being said, there are a few other additional features like pre and post flight checklists, mind mapping, notes templates, etc that YATTIE has. And, full transparency, the XRay tool has the JIRA integration that YATTIE doesn’t have yet.

In the end, I was looking for something open and more actively developed. It doesn’t seem like the XRay app has had any new features in at least several months, whereas I can now I can put all of the great ideas folks are throwing out directly into YATTIE and let the community decide what is most useful. (Like Richard’s suggestions on improving the reports!)

@oxygenaddict - Kampsite: Feature Requests, Feedback + Suggestions Board

It’s up next!

Please feel free add any other suggestions you have.

@oxygenaddict -

A new release with reporting summaries, hot off the presses. JIRA integration for quick uploading is next.

Thanks @simon_tomes (and Test Buddy) for the implementation idea! Would you happen to have an example of your old reports? I’m working through a sea of idea for improving reports and trying to distill it down to the most important pieces.

Sadly I don’t. Perhaps @alg might have some copies that he’s open to sharing. Al was a big fan of TestBuddy and helped us shape parts of the product.

That would be super helpful.

@alg - I’d love to hear your experiences with and thoughts about TestBuddy as well if you’d be willing to share!

@dacoaster I can share my experiences & thoughts of using TestBuddy if you’d find that helpful.

I’ll look at giving the latest version of Yattie a whirl in the new year.

Your comment on timelines is interesting. I never added that to Detective Vision as it wasn’t something that interested me. What I wanted was note taking as a starting point with screenshots and being easy to jump back and forth. In both Yattie and XRA I found the timeline to be no more than padding in the report. Perhaps part of that is because a timeline is best to show your journey but what I wanted (and what my team cared about) is the end findings.

Thank you for the offer @alg ! I’d love to hear about how you used the tool and what you found the most valuable!

Simon has also been more than gracious with his feedback and support!

I’ll send you a DM to set something up.

@oxygenaddict - I’m very excited to hear your feedback.

Re:timelines, I think they are most valuable in three scenarios:

  • When you are not as familiar with the product you’re testing. In this case they allow you to retrace your steps when you stumble upon an unexpected issue.
  • Mob testing or shared testing scenarios. So everyone can be on the same page and easily share results.
  • Training scenarios where a senior tester can review the “trail” of a more junior tester to provide help/feedback.

So, in some sense, a timeline seems like a “user friendly” place to start while working in the tool.

That being said, I also haven’t found it incredibly useful in my quick deep dive testing, so I’ve added an alternate “notes” view to create something more akin to “I have my Evernote open to stream-of-consciousness record what I’m thinking”. I actually JUST pushed this today - version 0.2.0 of YATTIE.

Coupled with the Summary at the end, it seems to be an improvement for that style of testing… But I’m keen to iterate on it with more usage and feedback. (I’m also looking to add a “Quick Test” button that skips the Test Charter page to facilitate this type of testing.)

Hi @dacoaster,

Congrats on your progress so far.

I’ve just added YATTIE to the Ministry of Testing Tools Directory.

Care to share what’s next on the roadmap and some of the challenges you’re facing and need help with?

Thank you @simon_tomes!

Right now we’re working on integrations => JIRA server, TestRail, and GitHub to start, but I’m always interested to hear from folks what would be most helpful for them.

From there, we’ll be looking to deepen the integrations and allow for creating new issues/bug reports/test executions/etc from directly inside YATTIE.

I’ve also had a number of conversations with folks around sharing test sessions with team members, so we’ll be working on making that easier.

Overall, the community has been incredibly helpful with folks giving detailed feedback and bug reports of course! Anyone looking to help out can just use the tool and let us know what they think. We also have a few ideas listed in our assistance section on GitHub, but feedback, taking our survey, and sharing the tool with others are always top of mind.

@simon_tomes - Thanks for the suggestion on iconography! It’s been added to v0.6.0 please check it out!

I have been looking a tool to report my exploratory testing and am starting to use Notion. Thank you for the suggestion