SWTC London - Heuristics and Oracles - June 2017


(Mark) #1

Thanks to everyone who attended the session, I was very impressed with how well you all managed to work out the rules for your board games and your contribution!

We would like to keep the conversation going by sharing other Heuristics out there and experiences of using them.

One that was mentioned during the session that I like to use a lot (especially for data driven testing) is the Test Heuristic Cheat sheet which is free to download and use.


(Chris) #2

Yo, Mark, I’m really happy for you, and imma let you finish, but HTSM had some of the best heuristics of all time. Some of the best heuristics of all time.


(Henry) #3

I do not know if the VETO or VEST patterns are mentioned any list of software testing heuristics.

VEST Verify, Enrich, Standardise, Transform

  • variant of the VETO architecture pattern
  • can be used independently for extract and load processes
  • can be a useful heuristic for thinking about tests for data transfer processes.

The VERIFY and TRANSFORM components are usually pretty obvious.
Consideration of ENRICH and STANDARDISE components can sometimes uncover less obvious and more subtle issues.

ENRICH (often a special kind of transform)

  • convert local lookup code or primary key to a physical unit symbol
  • ex - 143 > metres

STANDARDISE (often a special kind of transform)

  • datetime: BST > GMT?
  • datetime: EST > GMT?

Any conversion to a local standard physical unit could be thought of as part of the S from VEST.
(Incorrect assumptions about physical units resulted in an expensive spacecraft burning up in the atmosphere of Mars some years ago.)

I came across this pattern at a commodities trading company where it was used to design a systems integration toolkit. It became natural for the test team to think in terms of the pattern when creating tests.


(Mark) #4

@kinofrost absolutely agree on the Heuristic Test Strategy Model (which can be found here) just wanted to offer an alternative as the SWTC London students are getting sick of me going on about how awesome SFDIPOT is.

@hal-s those are some awesome heuristics and one’s I’ve not come across before. Thank you!

Anyone else got heuristics they want to share?


(Oleg Pantsjoha) #5

SFDIPOT is part of a bigger mnemonic, which is “cidtestdsfdpotcrusspicstmplfdsfscura”. What I’m currently looking for is a detailed break down for all of them. I found a nice top level break down here: https://www.infoq.com/news/2009/10/testing_heuristics

What I mean by a detailed break down is things that you need to think about for each heuristic, take S from SFDIPOT for example:

Structure. Everything that comprises the physical product.
 Code: the code structures that comprise the product, from executables to individual routines.
 Hardware: any hardware component that is integral to the product.
 Non-executable files: any files other than multimedia or programs, like text files, sample data, or help files.
 Collateral: anything beyond software and hardware that is also part of the product, such as paper documents, web links and content,
packaging, license agreements, etc.

It’d be nice to see that kind of break down for all 36 heuristics.


(Simon) #6

I can’t get enough of CRUD, it’s like a reliable friend!

“Create, Read, Update and Delete. These are the essential actions you can take on any data element within a system.” – as Elisabeth Hendrickson shares in her excellent book Explore It!.

Create CRUD combo power-ups with other heuristics! For example:

  • Combine “Never and Always” with “Delete” and “Update”: I should never be able to delete someone else’s embarrassing photo of me but I can always update the tag so others can’t see it!

  • Combine “Goldilocks” with “Read”: Those too big, too small and just right embarrassing photos should render with the same dimensions on my mobile device.

  • Combine “Interrupt” with “Create”: Kill your connection to the interweb before those embarrassing photos automatically make it online!


(Henry) #7

CRUD has been a good friend to me aswell. But really like your idea of combo power-ups.


(Dan) #8

Here’s a great mnemonic resource that I like to use:

It lists 35 different testing mnemonics that are useful for different purposes!


(Christian) #9

I simply love this table https://goo.gl/images/fjzL0f

Recently I have been introducing fcc cuts vids to people and they seem to like it a lot to get out of their comfort zone.


(Chris) #10

Never wear underwear for more than a day between washes.


(Oleg Pantsjoha) #11

The issue with this site though is that it doesn’t have a detailed break down of each heuristic. A tester who’s new to these won’t understand what they mean or questions they should be asking them selves when looking at them.


(Dan) #12

Very true, but it’s a good start for them to hear about the heuristic to drive more research :wink:


(Oleg Pantsjoha) #13

Courtesy of Paul Holland - dug up this article (by Michael Bolton) - http://www.developsense.com/articles/2005-09-StayingOnTheCriticalPath.pdf and Appendices (from Rapid Software Testing class) -
http://www.satisfice.com/rst-appendices.pdf

Which talk about some of the heuristics in detail. Great stuff!


(Oleg Pantsjoha) #15

Here’s a good article by James Bach about Oracles and type of testing we can do - http://www.satisfice.com/blog/archives/33