Power Hour - Approval Testing

Ministry of Testing MoT Power Hour
#1

Hi, I’m Emily Bache, a Technical Agile Coach and specialist in Approval Testing, I’m running a 99-minute workshop on Approval Testing. I’m here to answer your questions and give you an idea of what you can expect in the workshop. Feel free to ask me anything about Approval Testing. For example:

  • What’s the difference between Approval testing and assertion-based testing
  • How to get started
  • What tools are available
  • Situations where I’d consider this approach particularly suitable
  • Approval testing and its relationship to other testing approaches like data-driven testing, property-based testing, Behaviour-Driven Development etc
  • How to minimize automated test maintenance costs while still maximizing bug-finding potential

Get all your questions in by 6th August before 3pm GMT and I’ll do my best to answer them during my Power Hour!

99-Minute workshops are available to all Pro Members.

1 Like
#2

When is approval testing recommended, and when is it not recommended?

#3

How would you setup an approval test?

#4

What is approval testing?

#5

Hi Louise, lovely to hear from you.

I recommend you use Approval Testing instead of assertions in situations like this:

  • the output you want to check is some kind of multi-line string already. Especially if it’s json or xml.
  • it’s difficult to predict in advance what the correct value is, but you’ll know it when you see it.
  • you want to check several fields of a complex object. Instead of one assert for each field, write a printer that writes a readable, formatted string, for example with one field on each line ‘field1=foo’ and ‘field2=bar’ Use the printer output as the approved value.
  • you have a complex custom-built ‘matcher’ to compare the actual against the expected value. I’d be tempted to replace the matcher with a printer since a printer is often easier and less bug-prone to write.

on the other hand I’d be less likely to use approval testing if:

  • the value I want to check is a single value eg an integer or float
  • you have an accurate way to predict the value in advance and you’re writing the tests before the implementation

Regards,
Emily

2 Likes
#6

It’s a style of automated testing that has these characteristics:

  • Test cases check actual program output against a previously approved value, and any difference will fail the test.
  • Normally, a human inspects and approves some actual program output when creating a test case.
  • Design a Printer to display complex objects, instead of many assertions.
  • Approved values are stored separately from the sourcecode for the test case, although in the same VCS repository.
  • When a test case fails, you can use a tool to inspect the differences and update the approved value.
  • If actual program output is not yet available, the approved value may be a sketch of the expected output. You may need to compare it to actual output by hand instead of using a tool.

Regards,
Emily

2 Likes
#7

Hi Louise,

That question needs a little more context before I can really answer it precisely. Here are some suggestions of contexts and possible approaches:

  1. You already have an assertion-based automated test with several assertions. In that case, install the relevant language version of ‘Approvals’ (https://github.com/approvals) and replace the assertions with a call to Approvals.Verify(printout) where ‘printout’ is a string containing all the details you were previously asserting.

  2. You have a command-line program or REST API. Here I’d be tempted to use TextTest (http://texttest.org). It has good support for command-line programs. You write fixture code to set up the program, run it, and gather output as a string, or call the api and print the response.

  3. You have a web application. I’d look at Jest (https://jestjs.io/) or ReTest (https://github.com/retest/recheck-web). I have less experience with those tools, but both can be used for approval testing.

Does that answer your question at all?

Regards,
Emily

1 Like