Tonight weāll be joined by the wonderful @cakehurst for an Ask Me Anything session all about exploratory testing.
Iāll make sure to add any questions that we donāt get to and any resources mentioned during the session to this thread.
If you miss the live session, a recording will be available on the Ministry of Testing website for all Club level members once weāve edited it and added captions.
Have you got any more questions youād like to ask after? Add them here 
Mind map tools mentioned in the chat
Dan mentioned @lisihockeās talk in the chat
Lisiās blog has lots of info https://www.lisihocke.com/ you can also see her pairing posts A Tester's Journey: Testing Tour
@friendlytester wrote about heuristics
@maaret was hailed as Exploratory Testing royalty You can follow Maaret on Twitter https://twitter.com/maaretp Maaret has lots of resources on Exploratory Testing here - Exploratory Testing Index | Maaret PyhƤjƤrvi
Clumsy - clumsy, an utility for simulating broken network for Windows Vista / Windows 7 and above
Whenever I start at a new organization I set out a contract between myself and the team to bring them on the journey of āletās use exploratory testingā. This is something Iāve discussed in my blog post Why Iām talking to developers about exploratory testing. Explaining what exploratory testing is (and isnāt) helps people to be convinced as to its use and usefulness.
I also look to answer the following points:
Why should we do this?
We donāt need testing
Wonāt it slow us down?
What do we devs get out of it?
I always start by showing people that exploration is something they already do. I ask āWhen you get a new Phone / Game / App what do you do? Read the manual first or play with it to see what it does?ā If you use the product to learn about it, thatās exploratory testing!
Iāve run a number of practical workshops in the past where I work with people to show the basics of exploratory testing:
I also pair with people one on one to teach skills, show how to keep on track and share any heuristics (rules of thumb) that I use.
The simple answer for this is that a lot of the time I donāt until Iāve started looking. Because testing aims to find useful information for a team the amount of time that testing could take could be any amount of time whilst information is still useful.
In the real world we have deadlines and have to support our teams getting things ādoneā so I time box my testing. I will write test charters based on the risks I think are related to the feature (based on breaking it into components and then risks and tests) and will work with the team to prioritise and size these tests. Based on the priority and sizing I can say āOkay I plan to take this long for this feature, unless anything major comes up that we need to look into.ā
So really it comes down to working with the team to work out whatās interesting and useful for us to test and feed back any information that might require us to change these timings.
Yes I do! Break it down, just like you would for a scenario or a test case. When we think of too big of a system, we get bogged down in the e2e flow and canāt see what to test, so lets break it sown to make it easier.
Product > Feature > Components > Risks > Test ideas
How would you test a chair?
This is a standard question that we testers ask each other and our teams and we tend to find that we get about 5 or so answers before the ideas run out:
It can be hard to design tests without techniques and skills to help us to come up with lots of ideas quickly. One way we can do this is by thinking about the components of a product, rather than the whole product as one complete entity.
Break it down
Instead of thinking about the whole thing, we can break the product or feature into its component parts and concepts to help us come up with test ideas. Now youāre not testing a āchairā youāre testing ālegsā, ācolourā, ādesignā, āarmsā, āseatā, āmaterialā, etcā¦
From there we can look at the risks for each component and how they might not work.
Risks then lead to test ideas as we look to see how the product (or component) works in relation to that risk.
(Those images come from a blog post That I wrote for Bloom & Wild in 2020.)
Exploratory testing is using critical analysis and thinking to uncover and share information about a product, design or idea. That means unlike in scripted testing we donāt need a product to be able to start testing. We can start exploring ideas or designs as soon as we hear about them, questioning them to find out information thatāll shape how theyāre thought about, designed and implemented.
Responsibility lies ultimately with the team, the team wants better quality of their products so theyāre in charge of making sure that testing happens. As a tester in the team you might end up doing the testing (because you have the bests skills for it) and youāll certainly champion quality, but the whole team is accountable for quality.
Note: That doesnāt mean only testers should be doing exploratory testing. I try to teach my team mates how to do it through pairing so that everyone can start finding more information about quality.
Tooling will depend upon what youāre testing but fundamentally you need anything thatāll help you explore and find information about the product and a way of taking test notes.
Test Notes: Google docs, Excel, XMind, TestBuddy, Screen recorders, Audacity audio, Pen & paper, PicPick screenshot grabbers.
Networking: Charles proxy, Dev Tools, Clumsy network throttling.
APIs: Dev tools, Postman, SOAP UI.
UI: Colour marketing psychology, Accessibility tooling (WCAG 2.1, Axe, WAVE, Lighthouse, A11y)
Back end: SQL readers, IDEs, GitHub.
Hereās some screenshots of mindmaps that Iāve made in the past (I donāt have many examples because my work ones are not for public consumption 
).
here are some tests notes that show that Iāve captured details about the test session, a key to easily colour code the information and Iāve pulled out some issues. The notes themselves run left to right top to bottom as I explore the system. I mark them with colours to make it easy for me to call out questions and issues when Iām debriefing and sharing the information.
Exploratory testing is a testing technique that we use to interact with and learn about a product, design or idea with the express intention of sharing information about it. itās different from other testing techniques in that:
In my notes I tend to capture:
The idea of the notes is to have something that 1) I can use to share information about what Iāve looked at back to people 2) can be used to learn about how the product works for future training.
Iād say that exploratory testing creates more documentation and test notes than other forms of testing. If you have a specific need and know youāll be handing over a project to others then make your test notes more descriptive and verbose.
I also run debriefs and demos to share what Iāve learned about the product too.
@Barry E Iād be keen to understand more about the view that exploratory testing has a tendency for minimal documentation.
Of course you can be! Different people have different skills in testing based on the needs of their organisations. just because your organisation isnāt in a place where exploratory testing is needed (or can be utilised) that doesnāt make you a bad tester at all.
The same testing skills we use in all forms of testing (critical analysis, learning, inquisitiveness, asking questions, coming up with ideas, fearlessness, note taking, sharing information) are present across so many types of testing. Those are the things that make you a good tester, not the specifics of your technique implementation.
(However knowing exploratory testing will certainly help your information finding massively and will make you more marketable in the job market.)
Exploratory testing helps us find out information about what we donāt know. Once weāve run our exploration we have information that can be shared back to our automation capability to create scripted tests.
And remember that weāre not automating everything! Be selective about what needs to be automated from your exploratory testing results.
The outputs of exploratory testing is information that I share back to the team via my test notes and debriefs. From our debrief we may create fixes, bugs or new tests back on the Jira board, this is based on an agreement with the person Iām debriefing.
I store charters against Jira tickets for traceability and you can use something like Test Rail or Zephyr to help with that. Just write your charter as the the scenario and attach your test notes back to the scenario 9which is more than good enough to meet ISO standards of auditability).
Anything we learn from testing via exploratory testing can become a regression test in the same way any other test would.
I donāt create matrices or reports / stats for my exploratory testing because in my organisation thereās no call for it. Instead I provide demos and debriefs about the quality of the system and also share information at stand ups about how good what weāre building is.
āIāve looked at this endpoint, looking at all of the CRUD behaviours with different valid and invalid data types, including some naughty strings and 4 bit character inputs. Itās looking good, the responses are showing we persist the data in the database and the error handling for fields X, Y and Z are awesome with really meaningful error responses. Today I can find out more information about the structure of the JSON queries we send if we thing we need it?ā
rather than
ā59.00087% of tests have been run.ā
In the first instance the team knows whatās been covered and what else I might look into (and how good things are). they see that Iām engaged and trying to help them out and am working with them. In the second instance when I just report a number Iām not really saying anything about the quality of the system, Iām basically just telling people Iāve done something.
If people need a type of reporting at your organisation then Iād recommend creating a document like the below that will allow people to see what testing has been done, details of the quality (through bugs) and an indication of whatās left to run and how long thatāll take.
A good question. Exploratory testing is the learning of our system or product to share information about it. Because weāre not confirming behaviour, but instead learning about and playing back about what happens, we donāt need to really know about the system in advance.
Business knowledge can help to identify issues more easily, and shorten feedback time to seeing issues, but you can always debrief with a developer or product owner to see where business issues might arise.
Donāt fall into the trap of thinking that Exploratory testing is like an ad hoc smoke test, just unscripted regression to check what we already think should happen. If weāre confirming what we already know then weāre not really exploring the system and probably donāt need to use exploratory testing techniques for that testing; try scripted testing instead.
As a part of exploratory testing we should be creating test notes and sharing our findings about the system through demos, debriefs and the notes themselves. Regression tests will also be documented through our automation tests that we create.
If youāre in an environment where thereās a lot of new testers (or any team mates) joining your project then use that as an opportunity to make your test notes more verbose, so that they can be used for training.
Donāt fall into the trap of thinking that exploratory testing is chaotic, ad-hoc and not documented. We can plan our tests and charters with a scope and purpose, timebox them, prioritise them and create useful test notes for sharing the information we find.
I think inherently using your own life experience for something will involve bias. Asking āwill I like this?ā is leaning into your own bias completely. The trick is to know this, be humble and not use this as the only heuristic for your testing. Learn about other heuristics, share your findings with your team mates to see what they think, learn about your biases and also have empathy for your customers that arenāt like you.
Itās a little harder to think about what other people might want / like from a product, but itās totally doable. The more you learn about other people then the easier thatāll be too. Why not look into How Diversity and Inclusion can help improve testing?
The recording is now live too for anyone who wants to watch it back 
