Pairwise testing presentation

(George) #1

A week ago I had the honor of giving the keynote at IWCT 2018. The talk is an introduction to pairwise/combinatorial testing. Generally that means using a “small” number of test cases to cover all the interactions between inputs or configuration settings. (Tools automate the test case generation after some human analysis and judgment.) What’s new in the talk is automatic generation of cases to cover output equivalence classes, their boundaries, corners, etc. The main example starts on slide 22.
Adobe PDF
I welcome your comments and questions…

Embedded functions in combinatorial testing: Progress in automating test design
Abstract: This talk introduces combinatorial testing (CT), from its origins in design of experiments to its present role in verifying interactions in complex systems. A persistent CT usability challenge has been the definition and enforcement of constraints among test factor values. Research progress in this area has enabled increased adoption among practicing software engineers. Recent work at led to the introduction of an embedded functions feature in its CT design service. The feature allows functionally dependent relations among test factors to be defined as functions in a general purpose programming language, PHP. These relations enforce constraints among test factor values and insure that all valid combinations of determinant factors are available for the test design. Resulting usability improvements enable automated pairwise test designs to meet novel objectives: Cover equivalence classes of expected results; verify univariate and multivariate equivalence class boundaries; verify corners among intersecting boundaries and edges.