Pairwise Combinations

## The problem pairwise solves Configurable systems multiply. Take the quote path of an insurance quote engine with six configuration parameters: product line (4 values), payment cadence (3), rating region (7), prior-claims flag (2), sales channel (3) and discount bundle (5). The full cross-product is 4 x 3 x 7 x 2 x 3 x 5 = **2,520** distinct configurations. If one end-to-end quote run costs about 90 seconds, exhausting that matrix is roughly 63 hours of machine time. The configuration regression window inside a 3-week release train is one evening. Exhaustive testing is not merely expensive here; it is arithmetically unavailable. The naive alternative is to hand-pick "a few sensible combinations", which is unmeasurable: nobody can say what such a set covers or what it misses. Pairwise design gives that intuition a **criterion** so the resulting set can be counted, argued about and regenerated. ## The all-pairs criterion The criterion is: *for every choice of two distinct parameters, every combination of one value from the first and one value from the second appears together in at least one row.* Not every triple. Not every full configuration. Just every pair. Two counts make the idea concrete. The number of distinct pairs to cover is the sum, over all fifteen parameter pairings, of the product of their value counts. For the model above that is 232 pairs. The number of rows is bounded below by the product of the two largest value counts, because those 7 x 5 = 35 combinations must each occupy a different row. A generator produced **43 rows** for this model. Each row carries fifteen pairs at once, so 43 rows offer 645 pair slots for 232 distinct pairs — comfortable redundancy, and 1.7% of the cross-product. A set that satisfies this criterion is called a **covering array** of strength 2. "Strength" is the number of parameters whose interactions are guaranteed: strength 2 is pairwise, strength 3 is three-way, and strength equal to the parameter count is the exhaustive cross-product again. ## Why the reduction is so aggressive The leverage comes from reuse. A single row simultaneously supplies one pair for every parameter pairing, so pairs are covered in parallel rather than one at a time. Add a seventh parameter with 4 values and the cross-product multiplies by four — 10,080 configurations — while the covering array grows by only a handful of rows, because the new parameter's values can ride along in rows that already exist. The standard result is that covering-array size grows with the product of the largest domains and only **logarithmically** in the number of parameters. That asymmetry is the whole economic argument: parameters are cheap to add, wide parameters are not. ## How the set is built Three families of construction show up in practice. **Greedy row-at-a-time** methods repeatedly build the candidate row that newly covers the most uncovered pairs, then mark those pairs covered and repeat until none remain; it is simple, fast and produces near-minimal sets. **Parameter-order growth** methods start with a covering array over two parameters and extend it, first horizontally by adding a column for the next parameter and then vertically by appending rows for pairs that extension left uncovered. **Algebraic construction** derives rows from mathematical structures with strong balance properties. Minimising a covering array exactly is computationally hard, so real tools return a good set rather than a provably smallest one — which is why two tools, or two runs, can return different row counts for the same model. ## The empirical claim behind the technique, and its status Pairwise is usually justified by the claim that most failures are triggered by one or two interacting parameters, with only a minority needing three or more. Published studies of fielded systems do report shares in that direction, but the reported proportion varies substantially by domain and depends on how someone chose to model a "parameter" in the first place. Treat it as a useful default and a reason to *start* at strength 2 — not as a proven law, and never as evidence that three-way faults do not exist. A candidate who quotes it as a fixed percentage is overclaiming. ## What pairwise does not give you Three limits matter from day one. First, it does not choose the values a parameter contributes; that is decided before the array is generated, and a covering array over badly chosen values is a neatly organised waste of an evening. Second, it produces no **oracle** — the rows say what to run, never what a correct outcome looks like, and a suite that only checks for a crash will pass on configurations that quote the wrong premium. Third, it makes no claim about *order*: it covers which values coexist, not the sequence in which operations were performed. Interaction faults that depend on three parameters, or on ordering, fall outside a strength-2 guarantee by construction and have to be handled deliberately.