Tonight, @darktelecom will be joining us for a masterclass exploring exploratory testing (can you say that 10 times fast? )
If we donât get to your questions tonight weâll add them below. If youâd like to continue the conversation from the webinar, here is an excellent place to do that
Of course, the recording of the masterclass will be available to everyone in the Masterclass section of our website after.
Typically, Test Management Tools focus on âpass or failâ for test cases? How would you suggest to handle exploratory testing in such tools, where there is no longer this clear âpass or failâ?
1.1 Comment: Pass will hide the issues found. Fail will often ârequireâ to make it green in my experience.
1.2 Comment: In our pass/fail tool, pass means it has been explored and fail is the area couldnât be explored due to a blocker. This does mean most things âpassâ, however.
Could you describe how and with who you do the debriefing?
Do you have an idea how to help testers coming from a more traditional background to get started with Exploratory Testing and âdefend itâ against âmanagementâ that might be focussed on pass/fail statistics?
3.1 Comment: Invite them over to a pairing/ensemble session
3.2 Comment: I would also add to Thomasâs question if you have some quick wins tips in order to âsellâ Exploratory Testing more easily
What in your experience controls the kick-off of exploratory testing? Like is that driven off a system having new features or maybe done according to a regular schedule like once a quarter or some other period?
What is your opinion on doing exploratory testing in a pairing situation with developers? Is their knowledge of the software a curse?
Xray seems a useful tool! Thanks! Does it have the ability to save exploratory sessions for later playback and sharing across the team please? If so, is it just locally, to a file share or in JIRA?
If the system contains several websites and windows services - is it possible to record the test across these parts of the system in the same test run?
How long would you do an exploratory session for? Trying to timebox them makes sense in my experience as this kind of testing is demanding so keeping sessions reasonably short keeps you fresh. Would you agree please?
Well, as Iâve shown, quality assessment is different with exploratory testing; itâs a true assessment, so it means we got our own perception of quality based on what we explored.
With a traditional test management tool, you could for example create custom statuses to indicate an higher or lower confidence.
You can still mark it as FAIL if you have a blocking problem there.
Marking it as PASS, as youâve mentioned, can hide issues. But you can also create a PASS_NEEDS_DISCUSSION for example. I would advise you to try what works with your team. It also depends on how you share/debrief afterwards⌠you may want to have a more visible status to signal something or not.
Well, whenever debriefing itâs important to have there the people that were mostly involved in that feature/story/etc.
So, having the developers and testers makes all sense. In one of our teams, we do mob-programming with testers also involved. So feedback comes early. But later on, whenever exploring it in more depth, maybe we need to have the product owner involved. I all depends on the risks that were addressed. If some usability issue comes, maybe we need to bring one of the experts to the discussion.
In terms of single testers, in my experience doing the scripted testing => exploratory testing migration requires that people understand what theyâre doing, the limits of what theyâre doing, what they could achieve if done otherwise.
We need to understand why they have been working in that way⌠maybe they were told to do so, or simply they arenât aware that there are different and better ways.
Depending on the context, one thing we can try to do is to remove the inner details of scritped test cases⌠and start giving more and more freedom to the tester. At a certain point, people may reflect that they donât need that guide anymore and that the guide/script actually limits themselves to discover valuable information.
To management, I think we have to show them the limits of scripted testing and traditional test cases. How they miss unknown risks.
Pairing with them could be something interesting to try out. In that case, I would not say âletâs write some test cases and then letâs try exploratory testingâ. Instead, I would say âdo you have some time? I woud love do to an exercise with you and use your expertise to provide feedback on somethingâ⌠and then let that person explore somethiing and share thoughts with you. And then you could say âsee, this is precisely the kind of value we can get with exploring⌠we learn, together⌠find problems, find ideas, gapsâ.
Itâs important to understand where that person stands (needs and concerns) and format the message accordingly. What Iâve seen is that people wants numbers and green/red flags because that gives them an easy way to deal with information.
Another way to look at it, is that with scripted tests, even if the system and risks were static (which they are not), we focus on finding bugs by looking at our expectactions.
With exploratory testing we have the ability to further improve the product⌠because we raise concerns, new ideas and we also find gaps. Checks wonât find gaps. Itâs essential in a product to improve it by making sure we have there the things that matter and not just the things weâve implemented.
Every time is a good moment to explore Itâs not time that we waste, even if the product doesnât change.
Remember that risks are dynamic (team is dynamic, context is dynamic, relevance of certain features is also dunamic, etc). Therefore exploring always provides value (unless you do exactly the same).
If you have a big feature, for sure youâll need to explore it - and by explore I just donât mean âexplore after being builtâ; I mean it needs to be explored conceptually, discussed, explored whenever is being built and afterwards.
If youâre going to address a user story, then itâs good idea to explore it and at the same time implement some test automation for the happy paths, for example. Exploring complements automation (exploring also gives ideas for automation⌠and automation is also a way of exploring btw).
Whenever exploring we are walking in the unknow, and learning. If we postpone learning to somewhere is the future, the risk will still be there⌠the problem that itâs going to be harder to deal with it.
Each person brings a perspective. As I mentioned, and quoting once again professor Cem Kaner,
Use âdiverse half-measuresâ-- lots of different points of view, approaches, techniques, even if no one strategy is performed completely. â Cem Kaner, Black Box Software Testing
so each person has its own perspective, its own backpack of knowledge. Part of it may be overlapping, other not really.
Traditionaly, a developer has a greater backpack of knowledge in terms of the product, from a technical standpoint. Traditionaly, a tester has a greater backpack of knowledge in terms of testing.
Having developers and testers together, pairing, is for sure beneficial.
A conversation can start around the feature, from a funcional perspective, then go more to a technical side and back to the functional side.
In one of my teams, we have testers involved in mob-programming, being there to provide feedback and also do some guidance.
Well, just to clarify: there are 2 apps
What we call Xray (https://www.getxray.app/), that everyone knowns as such, is a test management solution that works inside Jira.
And now we have Xray Exploratory App (Xray - Exploratory Testing App) which is just a desktop app⌠that may connect with Jira+Xray, or you can simply use it without Jira+Xray
You can save your sessions locally, in a file. You can also export the session results to a .zip file, containining a PDF with all the notes and screenshots, plus the evidence as individual files.
If you do have Jira+Xray, then you can have some additional benefits such as uploading the results to Jira and also the ability to create defects right from the Xray Exploratory App.
Yes, what you do whenever exploring is up to you. In that case you would probably record the whole screen, instead of picking up a specific application and/or browser.
Whatâs important, is that you have a charter that gives you the mission and some limited scope. Otherwise, you can be wandering around as James Whittaker mention.
Exploratory testing without good guidance is like wandering around a city looking for cool tourist attractions. It helps to have a guide and to understand something about your destination. - James Whittaker
Itâs also better in terms of analyzing progress, because you can have a more limited and concrete scope that you can be able to provide a quality assessment.
you have that allocated time for handling the charter
it can also somehow implicitly enforce you to look more at relevant things first
you may compare sessions afterwards
etc
I would say that using 30m or 45m is fine but this is something that needs to be tried in the team.
One of my teams is not really timeboxing though; that is working for us until now.
You mentioned about being fresh; I totally agree, we need to focus and defocus to foster different types of thinking. Making a pause, walk a bit outdoor, even washing dishes (which works for me), can provide you new ideas you can try when you get back to the keyboard