Power Hour - Exploratory Testing

[simon_tomes]

@sharonoboyle

What are the fundamental areas we should really work on in order to become better exploratory testers?

An epic question, Sharon! And to be honest my brain has flat out stopped working after the power hour. Sorry I haven’t answered your question yet. I’ll reflect and reply soon enough. It’s such an excellent question.

[ezagroba (Elizabeth Zagroba)]

@sharonoboyle

What are the fundamental areas we should really work on in order to become better exploratory testers?

Mindfulness?

This seems like the wrong answer. But noticing things is the best thing you can do to be a better tester.

It will help you answer questions like:

• Have I seen this before? Under what conditions?
• This seems weird. What were the last four things I did before this happened?
• I have seen this before. Is there a pattern?
• What is the setup on my machine? How is that different from your machine?

[ezagroba (Elizabeth Zagroba)]

@simon_tomes

@nick.pass

Have you ever, or do you know of anyone who has, conducted exploratory testing in a regulated environment (banking, automotive, medical, etc)? What tips would you have for those who work in such sectors?

Hi Nick. Thanks for your questions!

I once worked as an exploratory tester at a software house that developed applications for the healthcare industry. And so did @ezagroba.

It was super regulated yet there were a tonne of talented exploratory testers across multiple sites. It was so cool to see this as there was a real drive to leverage the value of exploratory testing to uncover risks that test cases would unlikely not discover. Exploratory testing worked in conjunction with scenario based acceptance tests which were written by the exploratory testers and typically implemented as automated checks by engineers/developers.

Audit teams were keen to see the acceptance test scenarios as evidence of quality checks for regulation purposes. I can’t actually remember if they viewed exploratory testing notes or not. I don’t think they did, even though those notes were attached to the issue/project tracking software.

Some tips come to mind:

• With any change I think it’s important to establish what’s happening right now before jumping in with a vision to do something different, if indeed that’s a vision or goal
• Try the smallest exploratory testing session alongside existing testing approaches
• Attempt to share the value of exploratory testing as a partner to non-exploratory testing techniques. Sharing real-life examples will help. For example, “Hey, this set of scripted checks found these bugs and this set of exploratory testing charters helped us discover these unknowns and also the following problems. We also used exploratory testing techniques to review stories and system diagrams before we wrote a line of code. It helped us turn some implicit information into explicit scenarios for our acceptance tests.”

This is reminding me I discovered that the auditors did not look at our exploratory testing notes. And they didn’t need to be in a particular format. I was short on time, so rather than taking a million screenshots and writing up something about each of them (in a Microsoft Word document, which had been the default), I made a video with a voiceover. (This is not my area of expertise, so first I recorded a screen capture in Quicktime, then audio on my phone, and brought them together in iMovie. There’s probably a better way.)

The video made its way to our customer support team, who were thanking me months later for how useful it was to see someone clicking around and describe how something was supposed to work.

I think the moral of this story is: your documentation may end up in unexpected hands. Make it useful.

[sharonoboyle (Sharon)]

Thank you very much Elizabeth and Simon!

[akmathur (Ashish Mathur)]

So I could have been exploring for a while and while doing so, I hit upon a defect. Now what ? Maybe I do not understand the whole concept very well but I am wondering with a very manual nature to exploratory testing, what automation could reduce the efforts.

[conrad.braam (conrad)]

@akmathur

Maybe I do not understand the whole concept very well but I am wondering with a very manual nature to exploratory testing, what automation could reduce the efforts.

I am pretty much seeing this as a new starter, loads of things that look like bugs to me are in fact because I am nat familiar with some of the hardware (a certain fruit-based company comes to mind) , and because being fresh to the product, I spot tiny inconsistencies that a long-time user never would spot. So I enjoy being new to the bug-hunt, because I know I’m looking at everything with new eyes,

BUT, and I share your pain Ashish, I am often really lost and slow to figure out some of the functionality, because it’s not familiar territory. This makes me feel like I am covering very little ground in one day of testing. In comes the automation tooling, I prefer to call it “assistive” automation, use a automation tool to do the boring or time-consuming parts of your explore journey. Automating steps like how to pre-populate a database with dummy data will help you understand more of the system inner workings or structure and behaviours while you grow as an explorer. (I hope)

[murat (Murat Gurses)]

Hi.

Correct me if I am wrong. As far as I’m aware we perform exploratory testing when

1. We have limited time
2. When we have no/limited requirements/documentation.

Are these the only conditions and time where we can perform exploratory testing?
Can we perform exploratory testing when we have sufficient time and documentation?

I am new to exploratory testing and I would be happy if any of you can answer.

Thank you.

[simon_tomes]

Hi.

Correct me if I am wrong. As far as I’m aware we perform exploratory testing when

1. We have limited time
2. When we have no/limited requirements/documentation.

Are these the only conditions and time where we can perform exploratory testing?
Can we perform exploratory testing when we have sufficient time and documentation?

I am new to exploratory testing and I would be happy if any of you can answer.

Thank you.

Hi @murat,

Thanks for your question.

Yes, you can absolutely perform exploratory testing when you have sufficient time and documentation. I actively encourage it!

You can use exploratory testing techniques (and mindset) at any time. For example, imagine you’re looking at a requirement doc, you could run an exploratory testing session on the information in front of you. You might ask questions about some of the risks you’ve identified in the documentation.

Explore a wireframe. Explore a mockup. Explore databases, systems diagrams, APIs, acceptance criteria, ideas, processes, feature files, assumptions, the UI, specifications. Go and explore anything you can gather useful information from!

Time is an excellent constraint and I mostly time-box all my exploratory testing sessions (anything from 20 minutes to 90 minutes). It encourages focus, reduces information share overload and increases the likelihood of useful conversations sooner rather than later – and those conversations influence what to explore next.