When a defect class keeps recurring, how do you choose between a regression case, a static rule, and a design change?
answer
- Think of a ladder, not a test
- Eliminate, prevent, detect, observe, accept
- Signature decides rule versus case
- Every rung below the first costs forever
- Proof is the class stopping recurring
basics
~20 sPick the highest rung on the containment ladder the class can afford: make it unrepresentable by design, else catch it mechanically with a static rule if it has a machine-checkable signature, else pin one behaviour with a regression case. Accepting and monitoring is a legitimate fourth answer.
solid answer
~50 sTreat it as an investment choice across a ladder, from strongest to weakest: **eliminate** the class by a design, type or schema change so it cannot be expressed; **prevent** it mechanically with a static rule that fires at authoring or on the pipeline; **detect** it with a regression case that pins one concrete behaviour; or **accept and monitor** when the class is rare and cheap. A static rule needs a mechanical signature and a tolerable false-positive rate, or it gets muted and stops existing. A regression case buys certainty about one instance at a permanent run-time cost, so it needs a growth budget and a deletion policy. A design change has the highest leverage and the highest cost, and it is the only one that removes the class rather than watching for it. Choose from recurrence rate, blast radius and marginal cost - then verify the class actually stopped appearing.
code
pseudocode · 15 linesdecide(defect_class):
if recurrence < threshold and blast_radius small:
return ACCEPT_AND_MONITOR
if class can be made unrepresentable at bounded cost:
return ELIMINATE_BY_DESIGN # removes the class
if class has a mechanical signature
and expected false positives are tolerable:
return STATIC_RULE # reach across all teams
return REGRESSION_CASE at lowest level that holds the assertion
with: guarded_class recorded, growth budget honoured
# exactly one rung per class; verify by recurrence going to zerogo deeper
Know that a recurring defect class deserves more than a one-off fix, and that adding a check is only one of several possible responses. Being able to name the ladder is plenty here.
Be ready to say why a static rule needs a mechanical signature while a regression case can pin a behaviour, and to place a new case at the cheapest level that can hold its assertion honestly.
Show that you weigh permanent run cost against reach, defend a growth budget for the pack, and can argue against adding yet another slow case when a cheaper rung would do.
Own the portfolio view: which classes are worth eliminating, who owns a rule applied across teams, how existing violations are migrated, and how you prove the spend worked by watching a class stop recurring rather than by citing a cost ratio.
## The ladder When the same class of defect appears for the fourth time, the useful question is not "what test do we add" but **at which rung can we afford to stop it**. From strongest to weakest: 1. **Eliminate.** Change the design, the type, the schema or the interface so the defect cannot be expressed at all. An input form that cannot be misrepresented, an interface that will not compile with the mistake in it, a schema that rejects the shape at the boundary. This removes the class rather than watching for it. 2. **Prevent mechanically.** A static rule at authoring time or on the pipeline that fails the change. Cheap to run forever, catches instances that have not been written yet, and needs no test data. 3. **Detect cheaply.** A regression case at the lowest level that can hold the assertion honestly. It pins one concrete behaviour, and only that behaviour. 4. **Detect late.** An assembled, end-to-end case. Slow, more failure modes, but sometimes the only place the behaviour exists. 5. **Observe.** Field monitoring for the symptom, so the next instance is visible in minutes rather than days. 6. **Accept.** Record it as a known class, decide it is not worth the spend, and revisit if the rate changes. Every rung below the first is a permanent operating cost. That framing matters, because teams reflexively answer "add a test" and then wonder why the pack takes ninety minutes. ## When a static rule is the right answer A rule works when the class has a **mechanical signature**: something structurally recognisable in the artefact, independent of intent. Silent coercion of an unparseable value, an unchecked return, a forbidden call across a boundary, a missing timeout argument - all recognisable without judgement. A rule is the wrong answer when the class is only wrong *in context*. Then the rule fires on legitimate code, someone adds suppressions, the suppressions spread, and within a quarter the rule is decorative. Set an explicit false-positive tolerance before you write it, and treat "we will just suppress the false ones" as the failure mode it is. The other reason to prefer a rule is **reach**. One rule applied across every team catches the class in code nobody has written yet; a case in one component's pack protects one component. That reach is also its cost: a rule everyone must satisfy needs an owner, a migration path for existing violations, and a decision about whether it blocks or advises on day one. ## When a regression case is the right answer A case is right when the behaviour is **valuable to pin permanently** and the mistake has no structural signature - a specific boundary, a specific rounding rule, a specific handling of an odd input. It is also the honest answer when the class recurs in *behaviour* rather than in *shape*. The discipline that keeps this from degrading is a **growth budget**. A pack of a few hundred cases that gains one case per escape and never loses one becomes slow, then flaky, then ignored - and an ignored gate is worse than no gate, because it still costs money and now also costs trust. So: add at the lowest level that can hold the assertion; require every added case to name the class it guards; and review periodically for cases whose class has since been eliminated at a higher rung, which are then deletable rather than sacred. ## When the design change is worth it Elimination is the only intervention with a decreasing cost curve: pay once, and the class stops needing checks at every rung below. Justify it by **recurrence and blast radius**, not by a remembered cost multiplier. The claim that defects grow enormously more expensive the later they are found is directionally supported but the specific ratios in circulation are contested, and a principal who justifies a redesign with a made-up factor will be asked for the source. Argue instead from what is observable here: this class has appeared four times in two quarters, each instance cost a certain amount of field impact and repair, and the elimination costs a bounded piece of work. ## Deciding, and then proving it worked A workable rule of thumb: **recurrence** decides whether to act at all; **signature** decides rule versus case; **blast radius** decides whether to go up to elimination; **marginal cost** decides which level the case lands at. Pick exactly one intervention per class - a plan with an action at every rung is a plan that ships none of them. Then close the loop. The only evidence an intervention worked is that the class **stops appearing** in new defects, which you can only see if origin and escape were classified consistently to begin with. That feedback loop is the real reason to invest in the classification discipline at all: without it, prevention spending is a matter of opinion, and the loudest recent incident wins the budget.
- How do you keep a growing regression pack from becoming a slow, ignored gate?Budget its growth and its run time explicitly, add each case at the lowest level that can hold its assertion, require every case to name the class it guards, and review for deletions when a class has been eliminated higher up the ladder. A gate people routinely rerun or bypass has already stopped being a gate; treat its run time as a first-class constraint rather than an accident.
- A team wants a static rule that would flag roughly one legitimate case in five. Do you ship it?Not as a blocking rule. At that rate people suppress by habit and the rule becomes decorative while still costing review time. Options are to narrow the signature until the rate is tolerable, ship it as advisory with a review of the findings, or drop to a regression case for the specific behaviour. Decide the tolerance before writing the rule, not after the complaints.
- How do you justify a design change over a cheaper check to a sceptical stakeholder?With observed history rather than industry multipliers: how many times the class has recurred, the measured field impact and repair cost of each instance, and the permanent operating cost of the checks you would otherwise keep forever. Present elimination as removing a recurring liability, and be explicit that the late-defect cost ratios often quoted are contested.
- How do you know afterwards that the intervention worked?Watch the class, not the total. Consistent origin and escape classification lets you ask whether new defects of that class have stopped arriving and whether the escapes moved to an earlier, cheaper phase. If neither moved, the intervention addressed a class you did not actually have.
A road with repeated crashes at one bend can get a warning sign, a speed camera, or a rebuilt curve. Only the rebuild removes the hazard; the other two cost something forever and rely on someone still paying attention.
saying these in an interview costs you the question
- Answering every recurring class with another end-to-end case
- Shipping a blocking rule with an untolerable false-positive rate
- Assuming suppressions will stay rare and reviewed
- Planning an action at every rung at once
- Justifying investment with a quoted late-defect cost multiplier
- Never deleting cases whose class was eliminated by design