What's your pairwise testing story?

For testers who use pairwise or combinatorial test designs, and for those who want to learn more about it, the International Workshop on Combinatorial Testing (IWCT 2019) will be 23 April 2019 in Xi’an, China. I’m posting this notice specifically for testers – practitioners – because in my opinion testers have made some of the most interesting presentations in previous sessions.

The workshop offers an environment for academia and industry to collaborate:

  • to allow academic participants to learn about industrial experience in practical application of combinatorial designs to real-life testing, and

  • to allow industrial participants to hear about the latest advances in the field

Information about attendance and submissions is at the web site (below). Also, as an organizing committee member, I may be able to help with other questions.


IWCT 2019 will be held in conjunction with the IEEE International Conference on Software Testing, Validation and Verification (ICST 2019), 22-27 April. ICST provides a common forum for researchers and practitioners from academia, industry and government to present their latest research findings, ideas, developments and applications in the area of software testing, verification and validation.


Xi’an is one of the oldest cities in China, the capital of thirteen dynasties. It is the starting point of the Silk Road and home to the Terracotta Army of Emperor Qin Shi Huang.

First Call for papers IWCT 2019
7th International Workshop on Combinatorial Testing

April 23th, 2019, Xian, China

The IWCT2019 is to be held in conjunction with ICST2019 and will focus on combinatorial testing. The workshop welcomes both research submissions and industrial experience reports.

Important dates

Submission deadline: January 15th, 2019
Author notification: February 5th, 2019 (tentative)
Workshop: April 23rd, 2019

Submission Guidelines

Full and Short Papers: We invite submissions of high-quality papers presenting original work on both theoretical and practical aspects of combinatorial testing. We accept both full papers (up to 10 pages) and short papers (up to 4 pages).

Poster Session: We will also have a poster session, for authors to present their work in an informal and interactive setting. We encourage submissions presenting research work in progress or important conclusions from practical experience.
In particular, we welcome posters focusing on

  • experience reports from integration of CT by industrial organizations
  • lessons learned from using CT tools in industrial applications

An extended abstract of the poster (up to 2 pages) should be submitted for review by the submission deadline. Accepted poster abstracts will be included in the proceedings.

Proceedings and Best Paper Awards

o Accepted (full and short) papers as well as extended abstracts will be published in the IEEE Digital Library.
o This edition of IWCT will feature two best paper awards in the categories of “best foundation/modelling paper” and “best CT application paper”.


Topics of interest for full and short papers include, but are not limited to:

*** Combinatorial testing workflow
o Modeling the input space for CT
o Efficient algorithms to generate t-way test suites, especially involving support of constraints
o Determination of expected system behavior for each test case
o Executing CT test suites
o Combinatorial testing based fault localization
o Implementation of CT with existing testing infrastructures
o Handling changes in test requirements

*** Real-world experience in deployment of combinatorial testing
o Empirical studies and feedback from practical applications of CT
o Evaluation and return of investment metrics to assess the degree of usefulness of CT
o Methodology used for test space modeling and determination of interaction coverage requirements
o Discussion of challenges and open problems in the application of CT in industrial settings

*** Applicability of combinatorial testing
o Comparison and combination of CT with other dynamic verification methods
o Investigation of historical records of failures to determine the kind of CT which may have detected faults
o Combinatorial testing for concurrent and real-time systems
o CT for testing cloud computing systems and use of combinatorial methods in cloud architecture
o Application of CT in other domains, e.g. information security, study of gene regulation and other biotechnology applications, mechanical engineering, etc.
o Combinatorial testing of variability models for software product lines

*** Combinatorial and complementing methods
o Combinatorial analysis of existing test suites
o Test plan reduction and completeness
o CT and coverage metrics - combining the two, and studying the relationship between them


Organizing Committee:

  • Dimitris Simos, SBA Research, Austria (Program co-chair)
  • Yu Lei, Univesity of Texas Arlington, USA (Program co-chair)
  • Angelo Gargantini, University of Bergamo, Italy
  • Jacek Czerwonka, Microsoft Research, USA
  • George Sherwood, Testcover.com, USA
  • Rachel Tzoref-Brill, IBM Research, Israel

Steering Committee:

  • Raghu Kacker NIST, Gaithersburg, MD USA
  • Richard Kuhn NIST, Gaithersburg, MD USA
  • Itai Segall Nokia Bell Labs, Israel