Software testers often struggle with how to explain their exploratory testing techniques. At Ministry of Testing we believe if we can communicate what we are doing whilst we do exploratory testing then we can create a better and mutual understanding of the importance of our testing work.
Finding the right words isn’t always easy, so we reached out to the community (on Twitter, LinkedIn and Slack) to ask them how they explained their exploratory testing. We hope you find it useful and maybe you can add some comments to make the list even longer!
This is what they said:
By teaching classes on it
You set your goal or mission and timebox the session.
Then you are free on what and how you do it within the planned time, though you try to hit your goal or complete the mission during that time.
Just like play a game with time limit, your goal is to get you the next level.
You may go through different path and fight differently with the baddies.
Though, you always know fight is not the goal but survive and get to the next level. Especially near the end of a chapter
I explain Coverage, then gaining a workable knowledge of domain, and then cut to testing and its objective of finding important bugs.
Without coverage and domain knowledge - you cannot explore.
That exploratory is not ad hoc testing. Define time frame, domain of testing, objective and result. Without it its just ad hoc testing.
I explain the techniques that I use. I recently spent several days testing different parts of the app each day as a tourist.
We brainstorm areas outside of the requirements where code has been changed but not covered in requirements and areas that could be unintentionally impacted and we explore. If we find something, we make sure it’s covered on requirements that cover similar features in the future.
A themed ‘choose your own adventure’ with a countdown clock
Doing a demo
“Pull a chair, let’s explore together.”
‘Travel through in order to learn’ testing
If you have a formal explanation for an exploratory test, this is not exploratory enought as it should be.
Real life story about someone with no testing background:
For me exploratory testing is based on experience, curiosity, system knowledge, business knowledge, technical knowledge and discipline. But I usually direct the conversation to session based test management, especially in agile development.
Exploratory testing is not wandering around in the software, rather it is focused and intense targeted examination of specific elements.
Using my experience and paying attention to the way a user can use a system not how they should use a system.
By pairing or showing someone what I do and modeling my thought process
Either, I never bothered to read the requirements or design specs or
As I prefer, once having tested the functional flows for success, I DESIGN in tests to see how the system handles exceptions. I’m looking for the error handling messages or code to protect the solution from unintended consequences.
One of the best methods I found in recent years is Orthogonal Test theory. It has been around for years yet the maths was not easily documented. When introduced through a tool and practical training, it became really CLEAR how useful such techniques could be to maximise coverage and introduce exception testing yet reduce tests to a minimum.
In one project where we replaced 400 tests with 82, we applied additional exploratory tests around the defects to clarify what was causing the errors.
Model - investigate - analyze - adjust model and repeat
That perfect balance of knowns and unknowns usually does the trick.
Dumb & Curious
Exploratory testing is all about discovery, investigation, and learning. It emphasizes personal freedom and responsibility of the individual tester. It is defined as a type of testing where Test cases are not created in advance but testers check system on the fly. They may note down ideas about what to test before test execution. The focus of exploratory testing is more on testing as a “thinking” activity.
Exploratory testing -
- Is not random testing but it is ad-hoc testing with a purpose of find bugs
- Is structured and rigorous
- Is cognitively (thinking) structured as compared to the procedural structure of scripted testing. This structure comes from Charter, time boxing etc.
- Is highly teachable and manageable
- It is not a technique but it is an approach. What actions you perform next is governed by what you are doing currently
When we do it, we use the thinking hats!
The journey of discovery of the system under test. Encounter the application and explain what you see. Use whatever documents or other knowledge you have about you to help (such as requirements, bug reports, or conversations). Build your explanation, your model, as you try to find out all the functionality. As more and more functionality is discovered, you build up more detail in your model and can introduce more specific testing around those functions. This whole process is an alternation between interaction and analysis/explanation. The beauty and power of this process is how these alternating processes build upon eachother. Interaction builds analysis builds the model builds test ideas which increases scope of interaction and you repeat.
Exploratory testing is a discovery process with a high-level objective but the tester will adapt what they test based on what they learn during the test in terms of product and quality risks etc.
Techniques: Consider writing a test charter upfront defining test objective and test scenarios to be considered albeit you will deviate, time box it, buddy testing and record results using screen recording tools where possible.
Often the system may behave differently than expected but this may be down to integrations within today’s highly distributed I.T. environments which will only be seen (or often considered) when integrated and within context of its universe.
All testing are exploratory, if it is not exploratory then it is not a testing it is something else.
Experience and belly feeling
Exploratory testing to me means being able to see the unexplored or unusual paths/areas which we tend not to overview enough. Attempt manoeuvres let’s say a “frustrated user would do”, be creative and see all roads without entering noise. It can as well be a form of review of user experience and interface including all related functionalities.
It is same as riding a bike on unknown road to find out the question, where heck to go!
One of the best type of testing which I liked the most. I think it helps to find maximum bugs in the application. It covers functional and not functional requirements. It helps testers to motivate in every other possible way.
Start from one point and keep going with the flow which ends with application issues in hand.
I click on things, and the app crashes. How do I know where to click? Well, I ask developers about their biggest weakness and use that knowledge when designing the tests.