How do you explain your exploratory testing techniques?

Software testers often struggle with how to explain their exploratory testing techniques. At Ministry of Testing we believe if we can communicate what we are doing whilst we do exploratory testing then we can create a better and mutual understanding of the importance of our testing work.

Finding the right words isn’t always easy, so we reached out to the community (on Twitter, LinkedIn and Slack) to ask them how they explained their exploratory testing. We hope you find it useful and maybe you can add some comments to make the list even longer!

This is what they said:


By teaching classes on it


You set your goal or mission and timebox the session.

Then you are free on what and how you do it within the planned time, though you try to hit your goal or complete the mission during that time.

Just like play a game with time limit, your goal is to get you the next level.

You may go through different path and fight differently with the baddies.

Though, you always know fight is not the goal but survive and get to the next level. Especially near the end of a chapter


I explain Coverage, then gaining a workable knowledge of domain, and then cut to testing and its objective of finding important bugs.

Without coverage and domain knowledge - you cannot explore.


That exploratory is not ad hoc testing. Define time frame, domain of testing, objective and result. Without it its just ad hoc testing.


I explain the techniques that I use. I recently spent several days testing different parts of the app each day as a tourist.


We brainstorm areas outside of the requirements where code has been changed but not covered in requirements and areas that could be unintentionally impacted and we explore. If we find something, we make sure it’s covered on requirements that cover similar features in the future.


A themed ‘choose your own adventure’ with a countdown clock


Doing a demo


“Pull a chair, let’s explore together.”


‘Travel through in order to learn’ testing


If you have a formal explanation for an exploratory test, this is not exploratory enought as it should be.


Real life story about someone with no testing background:
Explaining exploratory testing with a table | Mindful tester


For me exploratory testing is based on experience, curiosity, system knowledge, business knowledge, technical knowledge and discipline. But I usually direct the conversation to session based test management, especially in agile development.

Session-Based Test Management - Satisfice, Inc.


Exploratory testing is not wandering around in the software, rather it is focused and intense targeted examination of specific elements.


Using my experience and paying attention to the way a user can use a system not how they should use a system.


By pairing or showing someone what I do and modeling my thought process


Either, I never bothered to read the requirements or design specs or

As I prefer, once having tested the functional flows for success, I DESIGN in tests to see how the system handles exceptions. I’m looking for the error handling messages or code to protect the solution from unintended consequences.

One of the best methods I found in recent years is Orthogonal Test theory. It has been around for years yet the maths was not easily documented. When introduced through a tool and practical training, it became really CLEAR how useful such techniques could be to maximise coverage and introduce exception testing yet reduce tests to a minimum.

In one project where we replaced 400 tests with 82, we applied additional exploratory tests around the defects to clarify what was causing the errors.


Model - investigate - analyze - adjust model and repeat


That perfect balance of knowns and unknowns usually does the trick.


Dumb & Curious


Exploratory testing is all about discovery, investigation, and learning. It emphasizes personal freedom and responsibility of the individual tester. It is defined as a type of testing where Test cases are not created in advance but testers check system on the fly. They may note down ideas about what to test before test execution. The focus of exploratory testing is more on testing as a “thinking” activity.

Exploratory testing -

  1. Is not random testing but it is ad-hoc testing with a purpose of find bugs
  2. Is structured and rigorous
  3. Is cognitively (thinking) structured as compared to the procedural structure of scripted testing. This structure comes from Charter, time boxing etc.
  4. Is highly teachable and manageable
  5. It is not a technique but it is an approach. What actions you perform next is governed by what you are doing currently

When we do it, we use the thinking hats!


The journey of discovery of the system under test. Encounter the application and explain what you see. Use whatever documents or other knowledge you have about you to help (such as requirements, bug reports, or conversations). Build your explanation, your model, as you try to find out all the functionality. As more and more functionality is discovered, you build up more detail in your model and can introduce more specific testing around those functions. This whole process is an alternation between interaction and analysis/explanation. The beauty and power of this process is how these alternating processes build upon eachother. Interaction builds analysis builds the model builds test ideas which increases scope of interaction and you repeat.


Exploratory testing is a discovery process with a high-level objective but the tester will adapt what they test based on what they learn during the test in terms of product and quality risks etc.

Techniques: Consider writing a test charter upfront defining test objective and test scenarios to be considered albeit you will deviate, time box it, buddy testing and record results using screen recording tools where possible.

Often the system may behave differently than expected but this may be down to integrations within today’s highly distributed I.T. environments which will only be seen (or often considered) when integrated and within context of its universe.


All testing are exploratory, if it is not exploratory then it is not a testing it is something else.


Experience and belly feeling


Exploratory testing to me means being able to see the unexplored or unusual paths/areas which we tend not to overview enough. Attempt manoeuvres let’s say a “frustrated user would do”, be creative and see all roads without entering noise. It can as well be a form of review of user experience and interface including all related functionalities.


It is same as riding a bike on unknown road to find out the question, where heck to go!


One of the best type of testing which I liked the most. I think it helps to find maximum bugs in the application. It covers functional and not functional requirements. It helps testers to motivate in every other possible way.

Start from one point and keep going with the flow which ends with application issues in hand.


I click on things, and the app crashes. How do I know where to click? Well, I ask developers about their biggest weakness and use that knowledge when designing the tests.


4 Likes

To reveal things which were not part of the test cases. Test cases cannot be exploratory. Rather, it shouldn’t be. Test cases should take care of all common scenarios, obvious product risks, while exploratory should take care of things which were not thought of during the test case development but worth finding before the client does. It is often confused with arbitrary or ad-hoc, but it is not. It is purposeful, but not performed with step 1, step 2 kind of procedures. It is imaginative within the requirements.

I’m very fond of quoting a comedy sketch by the British comedians Peter Cook and Dudley Moore. Moore is interviewing Cooke, who is in the role Sir Arthur Streeb-Greebling, the proprietor of a very unsuccessful restaurant.

“Have you learnt from your mistakes, Sir Arthur?”
“Yes I have, and I am confident that if asked, I could repeat them all perfectly.”

I normally use this in mockery of politicians who are fond of saying “we must learn from our mistakes” after any disaster or administrative failure. But having just this moment replicated a very difficult to replicate bug, and done it more easily the third time I tried, I think I shall add this to my repertoire of testing techniques!

“exploratory testing techniques” is a confusing combination of words
Exploratory testing is a style of testing.
Check Cem Kaner’s definition - who coined this term long long time ago.
See formality diagram suggested by RST: https://www.developsense.com/blog/2014/07/sock-puppets-of-formal-testing/
Also exploratory testing is supposed to be just testing(which already includes the exploration part): https://www.satisfice.com/blog/archives/1509
You can talk about testing techniques. Which can be applied in an exploratory manner or not.
See page 5 of Test Design course of Cem Kaner: http://testingeducation.org/BBST/testdesign/BBSTTestDesign2011pfinal.pdf
Or you can talk about exploration techniques or methods.

To give a few general examples of exploration methods in software testings:
reflect on what you know already, reflect on what you know you don’t know, use other people’s conversations on the matter you plan to explore to identify unknown unknowns, or review previews explorations in a different light or at a later time, create a map/leave some trails on where you went in your journey, know your oracles, and heuristics to help in your explorations.

Explaining the thought processes could be really hard, unless the other person knows exactly the same things you know.

Exploratory Testing is when you test so that you use learning about what you should test as you are finding it.

Thanks. Very well explained. I see your point on learning and using that learning to test better.

Implementing exploratory testing for user interface testing–

  1. Is not random testing but it is ad-hoc testing with a purpose of find bugs
  2. Is structured and rigorous
  3. Is cognitively (thinking) structured as compared to the procedural structure of scripted testing. This structure comes from Charter, time boxing etc.
  4. Is highly teachable and manageable
  5. It is not a technique but it is an approach. What actions you perform next is governed by what you are doing currently

During exploratory execution for user interface testing, the following needs to be done:

  1. The mission of testing should be very clear
  2. Keep notes on what needs to be tested, why it needs to be tested and the assessment of the product quality
  3. Tracking of questions and issues raised during exploratory testing
  4. Better to pair up the testers for effective testing
  5. The more we test, more likely to execute right test cases for the required scenarios

It is very important to take a document and monitor the following things in user interface testing:

  1. Test Coverage – Whether we have taken notes on the coverage of test cases and improve the quality of the software
  2. Risks – Which risks need to be covered and which are all important ones?
  3. Test Execution Log – Recordings on the test execution
  4. Issues / Queries – Take notes on the question and issues on the system

Smarter exploratory testing finds more errors in less time.