skip to content

A discovery workshop covered a utility billing tier rule, yet an off-by-one at the tier boundary reached production. How would you change how that workshop gathers examples?

level: seniorimportance: should knowfreq 41%

answer

  1. The agreed examples were all interior points
  2. Ambiguous words: above, at least, up to
  3. Below it, on it, above it, plus empty
  4. Say the number out loud in the room
  5. Write the resolved wording back into the rule

basics

~20 s

Make boundaries a standing obligation of the session: for every rule with a threshold, the group states the value below, at and above it, plus the empty case, and writes down which side the threshold falls on in the business's own words.

solid answer

~50 s

The workshop produced agreement about the typical case and left the boundary to be inferred, which is where the words in the rule do silent work -- 'above the cap' hides whether the cap itself is included. Fix the gathering, not the people. Give the session a short standing prompt applied to every rule with a threshold: state the case one unit below, exactly at, and one unit above it, plus zero and the largest plausible value. Require that the value at the boundary be written as a concrete example with the outcome spelled out, so someone must say the number out loud and the business can disagree. Treat any rule that ends the session with only one example as unexamined and take it no further. Then check that the boundary example survived into the acceptance criteria and the executable scenario, since a boundary agreed in the room and dropped at the hand-off fails in exactly the same way.

code

pseudocode · 10 lines
pseudocode
RULE (before)  usage above the tier-1 cap is billed at the tier-2 rate
  EX  320 units -> tier-1        # interior point, hides the ambiguity
  EX  780 units -> split         # interior point, hides the ambiguity

RULE (after)   usage of 500 units or less is billed at tier-1;
               usage beyond 500 is billed at tier-2 for the excess
  EX  499 units -> 499 at tier-1
  EX  500 units -> 500 at tier-1          # the boundary, stated
  EX  501 units -> 500 at tier-1, 1 at tier-2
  EX    0 units -> standing charge only

go deeper

for a junior

Remember that words like above, at least and up to hide an endpoint, and that a rule with a threshold needs examples just below it, exactly on it and just above it before anyone can claim to agree.

for a middle

Be able to show the mechanics: which examples the group wrote, why interior values hid the disagreement, and how the resolved answer gets written back into the rule's wording rather than kept in people's heads.

for a senior

Diagnose rather than blame. Explain what in the session's habits let the boundary go unasked, the specific change you would make, and how you would tell it worked -- disagreements surfacing in the room, not the absence of future escapes.

for a principal

Be ready to say what this class of escape costs beyond the code fix -- re-runs, corrections, support load -- and to resist quoting contested cost-multiplier figures while still making the case for spending the discovery minutes.

### What actually went wrong The workshop was not skipped and the rule was not missing. The rule said "usage above the tier-1 cap is billed at the tier-2 rate", and everyone in the room understood it -- differently. The business read the cap as inclusive: a customer who lands exactly on the cap stays on the cheap rate. The implementation read it exclusively. Nobody noticed, because the examples the group wrote were 320 units and 780 units. Both sides bill those two identically, so the disagreement was invisible. This is the characteristic failure of discovery: the session produces genuine shared understanding of the **typical** case and silent divergence at the **edge**. Prose hides it because the natural language of rules -- above, after, at least, up to, within, more than -- is exactly the vocabulary that is ambiguous about its own endpoint. Only a concrete value forces the question. The cost profile makes it worth engineering against. In one monthly **utility billing run**, the mis-priced boundary applied to 1,183 accounts before anyone noticed, and the correction was not a code fix but a code fix plus a re-run, plus re-issued statements, plus a support queue. The rating service in that run peaks at roughly 1,200 requests per minute, so the re-run itself is not free either. ### Changing the gathering, not the goodwill "Be more careful about edge cases" is not a change. The changes that hold are the ones that make the boundary question impossible to skip. **1. A standing boundary obligation.** For every rule that contains a threshold, an ordering, a count or a date range, the session must produce examples at: one unit below the threshold, exactly on it, one unit above it, and the degenerate case (zero, empty, none). This takes about ninety seconds per rule and it is the single highest-yield habit in discovery. **2. Make the boundary value be spoken.** The rule stays in business language, but the example must carry a number and an outcome: "500 units bills entirely at the tier-1 rate." A business representative who disagrees will say so instantly; the same person will nod along to "handles the cap correctly" every time. **3. Ban single-example rules.** A rule with exactly one example under it was illustrated, not examined. Treat it the same way as a rule with no examples: unfinished. **4. Let the testing perspective drive the second pass.** After the group has the typical examples, one person's explicit job is to walk each rule asking for the boundary, the empty case, the duplicate, the out-of-order arrival and the interrupted run. This is a role assigned in the room, not a personality trait, and it works even when it is the developer holding it that day. **5. Ask for the oracle.** For each example, ask how anyone would *know* the outcome was wrong. On the boundary case the honest answer was often "we would not, until a customer complained" -- which is itself a finding, and usually leads to a check on the billing run's output rather than just a scenario. **6. Confirm the example survived the hand-off.** A boundary agreed on a card and dropped when someone wrote the acceptance criteria fails identically to a boundary never discussed. It is worth a thirty-second check that each boundary example appears in the criteria and in the executable scenario, in the same words. ### The wording fix that pays for itself Once the group has answered "which side does the cap fall on?", write the answer into the rule: "usage of 500 units or less is billed at the tier-1 rate; usage beyond 500 is billed at the tier-2 rate for the excess." The ambiguity is now gone from the artefact rather than living only in the memory of the four people who were in the room. This is the cheap part, and teams skip it because the group already agrees -- which is precisely when the wording matters least to them and most to whoever reads it in six weeks. ### What to say about the incident itself An interviewer is listening for whether you blame or diagnose. The useful answer treats the escape as evidence about the *process*: the examples chosen were all interior points, no habit forced an endpoint, and the rule's wording preserved the ambiguity. Then say what you changed, and how you would know it worked -- for instance, that boundary examples now appear under every threshold rule in the last few stories, and that the next similar rule produced a disagreement in the room instead of in production. A finding at the board is the outcome you are engineering for; the absence of future defects is not something you can point at. Two things to avoid claiming. Do not assert a specific multiplier for how much more a defect costs in production than in discovery -- that ratio is widely quoted and genuinely contested, and its original evidence base is disputed. And do not promise the practice removes escapes; it moves a class of misunderstanding earlier, which is a real and sufficient claim on its own.

  • How do you keep this from turning into a checklist the group resents?
    Keep it to one prompt, apply it only to rules that actually contain a threshold or a range, and let the group see it pay. Ninety seconds per threshold rule is affordable; a twenty-item edge-case checklist read aloud is not, and it gets skipped within three sprints. The habit sticks when someone can point at a disagreement it exposed in the room last month.
  • Would more test cases after the fact have caught this instead?
    Only by luck. A test written from the same ambiguous rule encodes the same reading, so it passes against the wrong behaviour and adds false confidence. The defect is a disagreement about intent, and no amount of downstream case design settles intent -- it has to be resolved by the people who own the rule, which is what the workshop is for.
  • The business representative is not sure which side of the cap is correct. Now what?
    That is a question to park and chase, not to decide in the room by consensus. Record it as an open question with an owner, keep building the rules that are settled, and hold the affected rule out of development until the answer arrives. If a decision must be made to keep moving, write the assumption down as a rule so it is visible and challengeable rather than buried in the implementation.

saying these in an interview costs you the question

  • Blames the tester for missing an edge case
  • Answers only with more automated cases after the build
  • Leaves ambiguous wording like 'above the cap' in the rule
  • Accepts a single example as evidence a rule is understood
  • Quotes a fixed cost multiplier for late defects as settled fact
  • Agrees the boundary in the room but never checks it reached the scenario

context