How do you choose the interaction strength for a large configuration matrix, and defend the cost?

## Why strength is a budget decision Interaction strength *t* is the promise that every combination of values across any *t* parameters appears in some row. Its cost is not linear. Covering-array size grows with the product of the *t* largest value counts and only logarithmically with the number of parameters. For the quote-engine model — value counts 4, 3, 7, 2, 3, 5 — the strength-2 floor is 7 x 5 = 35 and a generator returned 43 rows. The strength-3 floor is 7 x 5 x 4 = 140, and generated sets land well above it. At roughly 90 seconds a row, 43 rows is about an hour and fits comfortably inside the configuration slot of a 3-week release train; a few hundred rows does not, and the honest consequence of setting strength 3 globally is that the suite gets skipped rather than run. That is the frame to bring to the interview: not "what strength is correct" but "what is the execution budget, and where does spending it buy the most exposure". ## The evidence, and how much weight it carries The usual justification for strength 2 is the finding that a large majority of failures are triggered by one or two interacting parameters, with the remainder needing three or more, and that the tail beyond a handful of parameters is thin. Published studies of fielded systems do report results in that direction. But the proportions vary substantially between the domains studied, and they depend on how a "parameter" was carved out of the system in the first place — a modelling choice, not a measurement. So use the claim as a **default and a prior**, not as a proof. A principal who quotes it as a settled percentage is overclaiming; one who ignores it and demands exhaustive testing is not doing capacity planning either. The defensible position is: start at 2 because the evidence points that way and the price is low, and let your own escape data update the prior. ## Mixed strength is the actual answer Uniform strength wastes budget at both ends: it over-tests parameters that never interact and under-tests the two or three that carry a real mechanism. The design that survives review is a tiered one. *Default tier, strength 2.* Every parameter in the model. Cheap, regenerated when the model changes, committed as a versioned artefact. *Elevated tier, strength 3.* A small sub-model, three or four parameters, chosen because a mechanism makes higher-order conspiracy plausible: anything spanning a partial write and its compensation, shared caches, feature flags that gate the same code path, or the parameters implicated in a defect that already escaped. In the quote engine, the parameters touching the pricing write, the document write and the compensating rollback are the elevated tier — that is exactly where a partial-failure rollback left orphaned quotes for four consecutive release trains under a clean strength-2 suite. *Seeded rows.* Configurations that must be evidenced by name — an audited setup, the largest broker's configuration, the last two production escapes. These cost almost nothing because their pairs count toward coverage. ## The inputs you should be able to name **Execution budget.** Rows times per-row cost against the window you actually own, including environment setup and the failure-triage time a larger set generates. **Escape history.** For each defect that reached production, reconstruct the interaction order that would have caught it. If escapes are consistently order 2, your suite is not the problem and more strength is not the fix. If several are order 3 in one subsystem, that names your elevated tier. **Blast radius.** Money, safety, regulatory exposure and reversibility. A subsystem you can roll back in minutes justifies less pre-release strength than one whose failure is a mis-priced policy sold to thousands of customers. **Model quality.** Strength cannot rescue a bad parameter model. If the values per parameter are poorly chosen or the constraints are stale, spending on strength buys thoroughly covered nonsense — fix the model first, and be willing to say so when someone asks for strength 3. ## What you own as a lead Three things beyond the number. First, **stated residual risk**: write down what strength 2 does not promise, so the decision is a documented tradeoff rather than an implied guarantee. Second, **stability**: the generated array is committed and regenerated on model change, not on every pipeline run, or failure comparison across releases becomes meaningless. Third, a **review trigger**: a rule such as "any production escape whose interaction order exceeds our strength in that area reopens the tier decision", which turns the choice into something evidence updates instead of something re-argued by whoever was burned most recently. ## The anti-patterns Setting strength 3 globally to look rigorous, and quietly disabling the suite when it overruns. Rotating a random subset of the cross-product each run so nothing is guaranteed and nothing is comparable. Deferring the full cross-product to a mythical end-of-train run that never happens. Treating the covering-array percentage as a quality metric to be maximised, when the actual quality question is what each row asserts.