Introducing Exploratory Testing to the Org

I’ve delegated for one of my team to trial using Exploratory testing instead of the usual test script approach for a couple of stories within the last couple of sprints. He’s really dived into it and has defined a Charter template and started running timeboxed sessions on certain areas. But the other testers in the team are less keen, they like the test case approach and are worried about how we ensure we hit all the acceptance criteria.

This is also a team where there isn’t a huge amount of automation, so there is a concern that if we do exploratory, we are going to miss more.

What tips and successes have people had when trying to bring exploratory formally into the ways of working?

5 Likes

First of all, you need to convince your team that Testing, by nature, is exploratory, irrespective of whether it is assisted with automation or not. This is a fact, and cannot be denied. A whole lot of discussions have already happened on ‘Test’ and ‘Check’, and everyone has agreed that automated checks by itself is not Testing.

To take care of the anxiety that you might miss out some acceptance criteria during human-led exploration, make sure that your charters cover them.

Also, Testing, at any point, is not just checking the acceptance criteria,. Quality is dependant on current software state. Only human-led exploration can lead to a satifactory acceptance of product Quality. The team needs to be educated on this.

All the best!

3 Likes

Exploring testing need not exclude the creation of scripted tests. Blend the two and you will get better scripts.

Exploring is great for new and great for change on existing systems.

Test scripts, if you avoid too much detail and implementation, are great for regression and weak for new features.

I always recommend and healthy and varied diet of testing, when I get the chance.

4 Likes

I wish I could be in your team (or any company) and learn exploratory testing/ET well. I am sick of the test case life. Anyway, I have done very little ET myself and not introduced it to a team. I only read one book on it and tried it on a few simple examples. But, I have some questions that might help you.

1 - Can you show your team the benefits of ET, perhaps by doing live sessions with them and hand holding them if needed? Perhaps you could start by doing a pilot for a small and non-critical feature.

2 - Is it possible to automate at least the crucial, time consuming scenarios so that the team has more time and confidence to do ET?

3 - If you can do (2), then can you hire a few automation experts to speed up the automation work?
I assume that you don’t have any people available to do this already.

2 Likes

Can you find the time to split your team into two, maybe just for one day, and have them test the same new changes independently and compare the results?

2 Likes

Before I even knew exploratory testing was a Thing, I was thinking of possible test scenarios that the scripting didn’t cover, and adding scripts for them. This included a number of possible failure scenarios (“How would the system perform if the user did X?”). The CEO later said “That’s the best-tested system we’ve ever had!!”

Unfortunately, it all went to pot when the system was released to beta test, because a) the requirements gathering that others had done was completely deficient, so my test assumptions fell short, and b) at the same time, Marketing were so impressed by the test results that they were bigging the system up to all our other clients and demanding we roll it out NOW. And then there was trouble.

Based on this experience and experience since, in retrospect I’d say I needed more exploratory testing, not less!

3 Likes

I’ve actually just written a draft for Ministry of Testing on this :slight_smile:
One thing that really helped at my old company was to introduce it as a trial - give people a chance to do it. (also can seem less scary than a “change”).

In terms of introducing it formally as a way of working, do you want eveyrone to do exploratory testing? or do you want to introduce it as an option? (but people can continue with test cases if they want?)

Also, do you think everyone at your organisation has the same understanding as you when you say “Exploratory testing”?

3 Likes

Good questions Nicola.

I certainly want it to be considered as an option but for the business to move away from always expecting 100s of test cases etc.

Exploratory testing does have different understandings, some just see it as an activity of “brief playing with the system” and some see it in the way it should be done etc.

1 Like

For a workshop I recycled an old text exercise. Participants were programmers, business analysts, and product owners.

It is basically: test the calculator.

1 Like

Once you have a shared understanding, then it might be easier to convince the business to be open to it.

The good thing with SBTM is that you can share findings in a more tangible, measurable way.

Do you have evidence of test cases being flawed that you can use to convince the business exploratory testing is a way to go? (e.g. number of test cases passed is misleading, you may have 95% pass but the state of the software is rather bad)