Are you doing pairwise testing today? If so, how is it going?

If the application you’re testing has more than one input field and those fields have more than one possible value apiece, you REALLY should be thinking about “combinatorial testing.” That’s a fancy phrase to describe a testing technique that covers just enough combinations of possible values for bug-finding power like you have probably never seen.

And if you are testing an application or device where safety or security is important, combinatorial testing is not just a nice to have. It’s probably an ethical obligation.

However, we software testers lack a reliable, open-source tool and a community to help establish combinatorial testing as a standard practice. Just like we made Ministry of Testing, let’s build a tool and community!

To find out more, check out my latest MoT article “Software testers unite: Let’s build a great tool and community for combinatorial testing!

What You’ll Learn:

  • What combinatorial testing is
  • Why it’s important
  • What tools are out there today to do it
  • What obstacles lie in the way of the widespread adoption of combinatorial testing
  • Why the testing community needs to start doing combinatorial testing as standard practice

After reading, share your thoughts:

  • Did you know that there are types of combinatorial testing that are more powerful than pairwise testing?
  • Which bugs have you encountered that were the result of a combination of two or more values across parameters?
  • Are you doing pairwise testing today? If so, how is it going?
2 Likes

I don’t know if you are aware, but James Bach published a pairwise tool 20 years ago. It’s not open source but it is free.
https://www.satisfice.com/download/allpairs

There is also Hexawise, but that’s not free. It can generate combinations of more variables (six, I think).
https://hexawise.com/

If you’re serious about doing pairwise testing, I suggest reading a paper that James Bach wrote on the topic. It’s very long and very heavy going - I confess I have read very little of it after the first couple of pages.
https://fuuu.be/polytech/INFOF307/Documentation/Pairwise%20Testing/Pairwise%20%2528negative%2529.pdf

Michael Bolton has also written at great length on the topic.
https://www.developsense.com/resource/pairwiseTesting.html

1 Like

Hi Steve,

Please do read the article, if you haven’t done so yet. It deals mainly with powers of combinatorial testing greater than pairwise.