Have you used Yattie? What were your experiences with it?

@parwalrahul has written us an excellent experience report around using the Exploratory testing tool Yattie:

It’s not a tool I’ve used before so I’d love to hear more about it. Have you used it? Do you like it? If you’ve not used it before, do you want to try it out?

5 Likes

I have never used a single tool for Exploratory testing. It is normally a combination of tools which makes YATTIE so exciting. Even though it is still in Beta form it is already a great tool to use in quick testing session or planned exploratory testing session.

I only spent 5 minutes with the tool and I already love it. Can’t wait to see how they will enhance the tool. Will definitely use it.

4 Likes

OK yes! This is awesome!!!

4 Likes

Actually looks good! Going to give it a try soon! :slight_smile:
Would love to see a demo or something or a tutorial video

hints @restertest :smiley:

3 Likes

Looks interesting. I might give it a go. Thank you @kristof

5 Likes

Excited to see YATTIE get more attention. Nice work, @dacoaster.

3 Likes

Looks interesting. I’ll test it out and make a report.

Edit: I did a report.

The potential is great. I use OneNote for almost all my testing, and one big drawback is that it’s basically a Windows-only note tool, so something that fulfills my notey needs in multiple operating systems is a good thing. I’m also all about the freedom of implementing notes, which is respected in this tool far more than other test-note-specific tools, which I appreciate a lot.

I ran two test sessions, one for installation, and one for reconning the functionality and there are three major factors I found holding me back from switching from OneNote:

Installation customisation

There’s no way to define where the installation lives. It lives on my C drive, which I don’t want, I have a hard drive for utilities to be installed to. I also don’t know where it stores screenshots and large video files before export, so I don’t know what’ll happen when my OS drive is full of video caching. I need this program to live on a different drive, or at the absolute minimum to default its export and caching on a different drive. In OneNote any file I make I can simply point a URI to on another drive.

Streamlining

I make notes on test ideas, bugs, issues a huge number of times a day. In OneNote I can tag these very easily with keyboard shortcuts, and add a checkbox for a “to-do” that I can tick off later so I don’t forget. I need adding “notes” to be very, very easy, which means keyboard shortcuts and UI integrations until I don’t feel like I’m stopping to click twice in the middle of my flow. I love OneNote for this (even more before they changed the tag system to shadow of what it was), so I need this tool to do it for me too. If I feel resistance to making these notes I’ll feel like I can’t do so.

I noticed there’s an open ticket in the tracker to add keyboard shortcuts, and the implementation of this (if it happens and how it’s done) will be critical in my decision to use the tool.

Data Safety

There’s no obvious backup of the session data before you manually make a save of it. No autosave or on-edit upload like in OneNote. This means that when I minimised the program and couldn’t get it to un-minimise again I had to force-close it, which lost any charter, notes, screenshots and videos I might have been recording at the time. If I lost an hour of testing I would be beyond furious. I need my note software to respect the value of my data and make useful saves and backups.

I have a bunch of other issues but I imagine they will be ironed out or fulfilled in time, especially around UI flow and design which shows a great deal of potential, especially around the Notes tab and adding notes. A little organisation and layout there and that’ll be a lot of value added over OneNote where I have to copy things under different subheadings sometimes.

3 Likes

Thank you @simon_tomes! The community has been very helpful and supportive and the tool has made some great strides. All of the attention and thorough feedback from amazing QA professionals is really giving great direction for the future!

3 Likes

Many many thanks for your thoughts @kinofrost!

Installation customization: I haven’t touched the defaults here, so I appreciate you calling this out and I will make sure to address it in one of the next few releases.

Streamlining: I am very interested in ease-of-use for power users, but since there aren’t any yet, I have been slow to start pushing these sorts of features. In your initial review, did you flip back and forth between the timeline and notes views? Or mainly stay on one? Do the current “to do” checkbox and label implementations resemble what you’re looking for, or is there something more or different you’d expect beyond just being able to tie them to a hot key? I don’t have any usage data on how people want to be using these things, so a lot of the feature work ends up just being me guessing. I will also look into how OneNote handles these things. I added your initial comments here for when I implement this.

Data safety: So all of the data lives in the installation folder by default. I’ve created a ticket here for allowing the app to recognize when there is old data in the folder and offer to allow one to resume where they left off.

I am actually currently working on the autosave/backup (along with better sharing) functionality, so I appreciate the importance here and should hopefully provide better coverage shortly.

Please do continue to share any feedback as you have the time or inclination - we try to be very responsive to help out as it is very much appreciated. We will definitely focus on these things to improve the overall experience.

2 Likes

Thank you for raising the tickets, which I forgot to do yesterday :grimacing:. I intend to get more feedback with more testing, when I can get both the time and the energy, so basically when my condition permits me.

How I currently work is I’ll write test notes like stuttered prose, stating what my current aims are, or briefly saying what my next experiment might be, and my thoughts on the validity of that, or further testing or risks or whatever. As I’m going I’ll put anything I want tagged on a new line, and press Ctrl+[a number] which represents a tag. I then go back afterwards and decide how to resolve everything.

I use tags for bugs, issues, important thoughts (includes test ideas) and questions. Bugs are problems with the software (“user input form is missing”), Issues are problems with the project (“I need access to the database that I don’t have”, “this needs automation hooks in it for me to test it better”, “this will take a long time we haven’t budgeted for”), thoughts/test ideas are usually risks or what I call risk-adjacent concepts that have to be resolved down, questions are stuff I need answers to that I usually need another person or other resource to answer. They can overlap, but I find that bugs are the most unique because that’s me saying “probably have to report this”. It doesn’t matter too much because I’ll go back and check them all at the end, and maybe already have answered my own question or found an observation to be invalid.

In the spirit of trying to be helpful, here’s some of my recon notes I did for the software, in OneNote, so you can see some examples. If you don’t want this out there let me know and I’ll remove it.

  • The exclamation mark is a thought about future testing. It tells me that there’s a whole area to test, but I’m asking myself if it’s a plugin (not “our” code, someone else tested it already so is lower risk, may be hard to fix if broken), or if you wrote it (higher test priority, fixable).
  • The checkbox I usually use for bugs, now, so that I remember to tick the box when I’ve investigated and raised/not raised the bug. Here I’ve noticed something about the usability of the UI, but I haven’t investigated it, tried workarounds, different settings, see if it’s tied to Windows, etc, so the box is unchecked.
  • I use screenshots to help, in line with my notes, so that I can make reference to them. Here I’m describing the button panel, and I can use these as subheadings for immediate further testing.
  • My question here is an expression of surprise, to remind me that I wasn’t expecting the behaviour, but also to find out how it’s supposed to work. I would have to go to the documents later and look it up, and find out if I’m just wrong about my assumption - the assumption is a useful discovery, even if I was wrong to assume it. It could have gone under important thoughts, or even bugs, but I find that careful categorisation of notes adds very little and acts as a distraction.

That’s essentially how I use notes and tags. Short paragraphs, and new things on their own line with a tag. When I’m doing this I go to a new line, type it out, use a keyboard shortcut for the tag, go to a new line, and that’s it; so my preference with the software would be the same thing in a different order - keyboard shortcut for, say, “bug” (problem with software), then write it out, then submit with a keystroke or shortcut, and move seemlessly on with my notes.

So that’s my flow. I want everything with absolute minimum of friction because as I write notes I’m also evaluating and problem-solving, and expounding ideas, especially in recon sessions where I’m super defocused and concentrated more on observing than detailed experiments. I use tags because I want to make sure I action something: bug investigation, inventing new charters, updating a risk catalogue, asking a question, informing someone of a problem, solving a project issue, think about the causes of bad emotional reactions, follow up on confusion with more research and testing, whatever. The notes are what I thought and did, the tags say “do something about this, sooner or later”.

Hopefully that’s of some help!

2 Likes

I have not used session helpers like these really, I have tried couple Chrome addons but nothing that ever that held my interest beyond initial trial.

I will share this with the colleagues, see if anyone gives a run and want to contribute, sharing link to this thread too.

Another thing that I have shared and discussed at the work is tools that help you capture logs with screenshots etc and create for example tickets out of them. But due to the security aspects none can be rolled out to any of our client cases. See example https://jam.dev/

1 Like

Thank you for this @kinofrost! I think this gives me what I need to at least take a first pass at implementation. I’ll make updates on the GH ticket as I am able to get on it.

1 Like

Thank you for the thoughts @finnish.

I have definitely thought about how we can include more information in the creation of issues, but there is an additional wrinkle of complexity here since the tool can test more than just web apps and lives outside the browser’s sandbox. I’ve done some research on how we might overcome some of those items short-comings but have yet to create a ticket for it because I’m a bit uncertain about what would be the most practical and helpful for real users. So this feedback is very useful.

Here is the link to join Yattie’s Discord Community: YATC (discord.com)

2 Likes

@kinofrost - We pushed updates to allow for installation directory configurations and the various dark mode UI issues you documented.

I’m working on autosaving, data recovery improvements, and upgrades to hotkeys/note taking next!

1 Like

Awesome :3. I’m going to try to take another look soon, write up some findings and maybe put them on the Discord for feedback.

@kinofrost - v0.8.0 is out with fixes around session persistence, recovery, and caching.

Many thanks for your feedback on the issue!

1 Like

@kinofrost - I know it’s been a few weeks, but I’ve almost got hotkeys ready to ship as well. v0.10.0 should be out by the end of next week with hotkey support!

2 Likes

v0.10.0 went out yesterday with hotkey support!

2 Likes