Test Case Matrix

Does anyone have a good way/tool to build out case matrices?

e.g. in a fictitious ERP system that closely resembles one I’m working on

  • specify a product to order (dimension 1)
    • subcategory of product: run through different logistics processes (dimension 2)
  • change order during any logistics stage has different outcomes (dimension 3)
    • some logistics stages have sub-stages. e.g. delivery has pick, pack, drive, unload. I suppose this could just extend dimension 3
  • some cases aren’t possible (e.g. some products can only haven order qty of 1 - and that de-prioritizes the rest of the child tests related to that one).

Do you just use spreadsheets/subtables? or is there a better way?

4 Likes

Just can speak about using spreadsheets for tough stuff like you described :sweat_smile: Everyone knows spreadsheets are easy to use and good for lists, but what really got me was how you can do more complex stuff with the right setup. Throwing in tables, some smart formulas, and combinatorial techniques can turn a basic spreadsheet into something way more powerful (but I’m not an expert in doing this :smiling_face: ) You can sort through and make sense of complicated data. You can handle lots of different info all in one spot without getting lost in the details using some specialized test management tools together with mind mapping and diagramming tools :slight_smile: maybe spreadsheets are a bit old-fashioned and aren’t trendy but it works perfectly in many situations

3 Likes

It depends on your reason for having a matrix, but generally I would use a combination of your test case management tool (ALM, XRay, etc) and Excel or Project.

In the test management tool build a library of reusable test step components at a transaction / function level (eg picking), and link them together to form longer test scenarios (order to dispatch, etc). Level 1 = scenario, level 2 = transaction, level 3 = test steps.

In Excel or Project, import your level 1 scenario (order to dispatch) and level 2 transactions. Against your level 2 tcodes you can then map security roles, and against roles you can map testers; you can add estimated execution times, etc. Against level 1 scenarios you can map your risk assessment prioritisation and expected start date for planning.

Once that’s done, a few pivot tables or views will give you resource allocations, a schedule, highlight overlaps, etc

2 Likes

The comments so far are helpful, thank you!

A few clarifications:

  • I’m not looking to build out full test plans with steps. I just want to identify input combinations and do some basic annotations/filtering to that matrix.
  • The goal is to consider and list out all the possibilities. Then to identify impossible cases, do some basic risk analysis and prioritization.
3 Likes

you really want some kind of graphical tool that allows you to specify the boundary cases while dropping down the steps so that where boundaries for a set of steps can lead to any incompatible/bottlenecking/confusion combinations in the assembly line and heatmap these easily using UML like symbols. Perhaps a UML tool is your best bet?

1 Like

I wish there was a better way. The benefit of spreadsheets is that they are a common interface between the business and engineers. The problem is that to make our systems testable it requires more constraints that a spreadsheet can support. If the business is changing constantly, stick with spreadsheets.

Decision model and notation and business process model and notation are a standard that might help in a case.

I’ve worked on business rule engine systems and making a visual test is hard to grok for hundreds of combinations.

1 Like

Totally agree - there is a case for just absolute accessibility and visibility as well. Spreadsheets transcend testers, developers, stakeholders - heck you can even send a spreadsheet to a totally non-technical person and they’d get the gist of it.

I would try a two-dimensions state-transition_table like this:

Maybe a orthogonal array tool (I’m not sure for the name) can map out all combinations.

1 Like