skip to content

A defect only reproduced when three conditions coincided. How do you write the explanation so the team can prevent it?

level: seniorimportance: nice to knowfreq 27%

answer

  1. Several conditions, one assumption
  2. Join the middle layer with and
  3. Mark each member ordinary or rare
  4. One named cause hides the other combinations

basics

~20 s

Write the conjunction, not one cause. Record all three conditions joined by and, marking which are ordinary and which rare, then name the single assumption they jointly broke. Breaking one condition stops this occurrence; only the weakness stops other combinations.

solid answer

~40 s

When several conditions had to coincide, the layers still hold: there are simply several entries at the middle layer, joined by *and*. Write them out, and mark which are ordinary states that will be true again and which are genuinely rare. Then ask the third-layer question of the whole set: what assumption did the code hold that these conditions together broke? That is usually one thing, and it is the only part that generalises. Promoting one member to "the root cause" is misleading in a specific way: it makes the others look incidental, so the team removes the named one and leaves a system where a different combination reproduces the same failure. Breaking the cheapest member is still a good move for relief, but record it as that, and say what remains reachable.

code

pseudocode · 10 lines
pseudocode
occurrence requires: A and B and C
  A: cache entry being replaced mid-read      (ordinary, many times a day)
  B: retry arrives before the write settles   (load-shaped, grows with traffic)
  C: row has no owner, from a 2019 import     (rare, removable)

weakness: the read path assumes an entry is present or absent,
          never being replaced -> no handling for an in-flight state

shipped:  removed C (hours). A and B remain ordinary.
open:     weakness; any new condition can pair with A and B again

go deeper

for a junior

Know that some defects need more than one thing to be true at the same moment, and that saying so is not a failure to find the cause. Record every condition you needed in order to make it happen.

for a middle

Explain why removing one of several coinciding conditions stops this occurrence but not the others, and how you would write the combination on the defect record so it is usable later.

for a senior

Show that you look underneath the combination for the single assumption it broke, and that you separate stopping the coincidence cheaply from closing the weakness that made the coincidence dangerous.

for a principal

Own the expectation that some explanations are conjunctions. A culture that demands one root cause per defect gets confident records that prevent nothing, and nobody notices until the same failure returns by a new combination.

## Some defects have no single root Many defects have exactly one condition: an input the code could not handle, with everything else about the occurrence incidental. Some do not. A failure that appears only when a cached entry is being replaced **and** a retry arrives **and** the row it touches has no owner is not a defect with one cause and two details. Its middle layer is a conjunction: A and B and C, all true in the same moment. The pull towards reducing that to one cause is strong, because a single-cause sentence sounds like understanding and a three-part one sounds like an excuse. It is the wrong instinct here, and the damage it does is specific. Naming one member as *the* cause makes the other two look incidental, so the team removes the named one and leaves a system in which a different combination reproduces the same failure. ## The layers still hold - **Symptom.** Unchanged. What was observed is still one observation. - **Condition.** Now a *set*, written with **and** between its members, each with the evidence that it was true at the time. - **Weakness.** Usually still one thing, and this is the important part. The conjunction is what the code failed to consider, and it failed to consider it because of a single assumption. "A read sees an entry as either present or absent, never as being replaced" accounts for all three conditions at once. So the explanation is not three parallel investigations. It is several conditions and the one assumption they jointly broke. If no single assumption emerges, either you have not gone deep enough, or you genuinely have two defects that happened to share an occurrence. ## Sort the set before you write it Not every member of a conjunction is equally interesting: | Kind of condition | How to recognise it | What it means for prevention | |---|---|---| | Ordinary | The system's normal state, true many times a day | Cannot be removed; the weakness has to tolerate it | | Load-shaped | Appears as concurrency or volume rises | Will recur, and more often as the product grows | | Rare | An old bulk import, a partial write, a hand-edited row | Removable, and often the cheapest member to break | That sorting is what tells you which member to break for relief and which ones you must design around. A conjunction with two ordinary members is far more dangerous than its apparent rarity suggests, because only one unusual thing has to happen for it to fire again. ## Breaking the conjunction is a fix, not an explanation Removing the cheapest member, usually the rare one, stops the failure today, and that is a good move. It is a condition-level fix with a useful extra property: because the failure needed all three members, breaking any one of them is enough for now. But it inherits the usual limitation of that layer. Tomorrow a fourth condition can combine with the two ordinary ones exactly as the rare one did, because the assumption those ordinary conditions broke is still held. The record has to say which of the two was done, and what remains reachable. ## Verifying a conjunctive defect Re-creating a coincidence in a controlled test is often harder than the fix was. Two workable approaches: 1. **Drive the members together deliberately.** Force the replacement window open, force the retry, plant a row without an owner, and confirm the observed symptom is gone. Expensive to build, but it proves the whole path. 2. **Verify the invariant instead.** Assert directly that the code now handles the state it previously assumed away, however that state arises. Cheaper and more durable, and it does not depend on reproducing something rare, but it only demonstrates that the weakness is closed, so it belongs with a weakness-level fix rather than with a broken conjunction. The wrong answer is the tempting one: exercise the single condition you named as the cause, watch it pass, and conclude the defect is gone. That check was never able to fail, since the other members were not present. ## Why one cause gets named anyway Three pressures, worth recognising in yourself: - **The shape of the record.** A single cause line invites a single sentence. - **The audience.** One line is easier to report upward than a conjunction with three members and a caveat. - **Hindsight.** Once you know the answer the rare member looks obviously abnormal and the ordinary ones fade into the background, because they were true all along and therefore stop feeling causal. None of the three is evidence about the system. ## What the record should say Write the conjunction out in full. Mark each member ordinary, load-shaped or rare. Name the single assumption they jointly broke. State which layer the shipped change addressed, and say plainly what is still reachable. An explanation naming one cause reads better and prevents less, and a team norm demanding exactly one root cause per defect will always get one, confidently written, and wrong whenever the world was more complicated than the form.

  • Is it ever right to name one of the coinciding conditions as the cause?
    Only when one is genuinely abnormal and the others are the system's ordinary state. Then the abnormal one is the condition and the ordinary ones are the environment the weakness lives in, and you say that explicitly. If two or more members are abnormal, the honest record is a conjunction, because removing one still leaves the other combination reachable.
  • How does a conjunction change what you do to confirm the fix?
    Exercising the one member you named proves nothing, because that check could never have failed on its own. Either drive all the members together deliberately and confirm the symptom is gone, or assert the invariant directly, that the code now handles the state it previously assumed away. The second is cheaper and does not depend on re-creating something rare.

A fire needs fuel, heat and air together; naming the match as the root cause tells you nothing about the paper stacked against the heater.

saying these in an interview costs you the question

  • Promotes the rarest condition to sole root cause
  • Calls the other conditions incidental and drops them
  • Assumes a rare condition will not arise again
  • Confirms the fix by exercising one condition alone
  • Insists every defect must have exactly one root cause