skip to content

How can an off-by-one defect survive a suite that already contains boundary tests?

level: seniorimportance: should knowfreq 54%

answer

  1. Where did the edge come from?
  2. The test can agree with the bug
  3. Wording that never says in or out
  4. Two rules claiming the same value
  5. Asserting no error is not an oracle

basics

~20 s

Usually because the boundary cases were derived from the implementation rather than the written rule, so they agree with the bug. Ambiguous wording, two rules claiming one edge value, and a weak assertion produce the same green result.

solid answer

~40 s

The most common cause is that the edge cases were written by reading the comparison out of the code, so the test encodes the same wrong limit the implementation uses and passes forever. Second is ambiguous wording — a rule that says between or up to without saying whether the end is included — which lets test and implementation take opposite readings. Third is overlapping rules: when two rules both claim the value on the shared edge, the outcome depends on which is evaluated first, so a case written against the current order proves only that order. Fourth is a weak oracle, asserting that the edge value was accepted without asserting what it produced. Fifth is a limit that applies to a value the system derives, which no input field exposes.

code

pseudocode · 13 lines
pseudocode
bands = [
  band(from=0,  to=40, multiplier=1.00),
  band(from=40, to=60, multiplier=1.50),
  band(from=60, to=INF, multiplier=2.00)
]

multiplier_for(hours):
  for b in bands:                 # first match wins
    if b.from <= hours <= b.to:
      return b.multiplier

# 40.00 matches two bands; the answer depends on list order
assert multiplier_for(40.00) == 1.00    # states the rule, not the order

go deeper

for a junior

Remember the core trap: an edge value copied from the code makes the test agree with the bug. Take limits from the written rule, and ask when the wording does not say whether the end value is included.

for a middle

Be able to walk the causes — implementation-derived edges, ambiguous wording, weak assertions at the edge — and show that a boundary case asserts the exact outcome rather than the absence of an error.

for a senior

Diagnose from an incident: restate the rule, establish where the existing edge came from, look for a second rule claiming the same value, and check what the case actually asserted. Talk about limits on derived values, not just on form fields.

for a principal

Own the systemic fix: rules that never let two of them claim one value, edges traceable to a written source, and a review habit that treats an ambiguous limit as a defect in the specification rather than a judgement call for whoever writes the test.

## The question behind the question An interviewer asking this is not testing whether you know what an off-by-one is. They are testing whether you understand that a passing boundary test is only as good as the source the boundary came from and the strength of the assertion at the edge. There are five recurring ways the technique is performed correctly and still finds nothing. ## 1. The edge was read out of the implementation This is the dominant cause. Someone opens the code, finds the comparison, and builds cases around whatever constant is there. The test and the implementation now agree by construction, and the pair will stay green through every refactor. The only way the defect surfaces is when a person compares the behaviour with the written rule — which is precisely the comparison the test was supposed to automate. The discipline is dull and effective: derive each edge from the specification, write the case, and only then look at the code. Where the specification is silent, that silence is a finding to raise, not a gap to fill from the source. ## 2. The written rule is ambiguous *Between 1 and 60*, *up to 500 records*, *no more than a fortnight* — none of these says whether the end value is in or out. A tester reads one way, an implementer reads the other, and both are defensible. The suite then enforces a reading that the code does not share, or, worse, the tester quietly adopts the code's reading to make the case pass. Ambiguity resolved in a test file is not resolved; it belongs in the rule. ## 3. Two rules both claim the edge — the ordering assumption This one is subtle and it is where a real incident usually comes from. A payroll engine bands weekly hours to pick a supplement multiplier, and the bands were written as: 0 to 40, 40 to 60, 60 and above. Notice that 40 appears in two bands and 60 appears in two bands. The engine evaluates bands in list order and takes the first match, so 40.00 lands in the first band and 60.00 lands in the second. The boundary suite was written the same way — by walking the same list top to bottom — and every case is green. Then a change reorders the band list, for an unrelated reason that looks entirely cosmetic. Now 40.00 matches the second band first, and every employee sitting exactly on the standard-week edge is paid a supplement they should not receive. The boundary tests still pass on the new order too, because they were only ever asserting *some* band matched. The defect was never in the comparison. It was in a specification that let two rules own the same value, plus code that resolved the overlap by evaluation order, plus tests that inherited that order as though it were the rule. The lesson is that when an edge value is claimed by more than one rule, the boundary case must assert the specific outcome the written rule demands for that exact value, and the overlap itself should be reported as a defect in the rule set rather than papered over. ## 4. The oracle at the edge is too weak A case that submits 61 hours and asserts only that no error was thrown, or that a page rendered, is not testing the boundary. It passes whether the value was accepted, rejected, silently clamped or rounded down. Boundary cases need the sharpest oracle in the suite because they are asserting a one-step difference — the outcome, the message, the computed figure, not merely the absence of an exception. ## 5. The boundary is not on an input Many real limits apply to values the system derives: a rounded gross figure, a count accumulated over a batch, a proportion computed from two other fields, an index into a table. No form field exposes them, so a suite built by walking the user interface never reaches them. Finding these requires reading the rules for the limits they state about intermediate quantities, then reaching them through inputs chosen so that the derived value lands on its edge — which is often several steps of arithmetic away from anything a user types. ## How you answer the diagnosis version If the interviewer frames it as an incident rather than a quiz, work backwards. State the rule as written. State what the system did at the exact value. Ask where the existing case's edge came from — specification, code, or memory. Check whether any other rule claims the same value. Check what the existing case actually asserted. That sequence usually lands on one of the five in a few minutes, and it separates a candidate who has debugged a live edge case from one who has only read about the technique.

  • You inherit a suite whose edge values you suspect were copied from the code. How do you check without rewriting everything?
    Take the rules that carry money or eligibility consequences, restate each limit from the written rule, and compare the restated edge with the value in the existing case. Disagreements are either a defect or an ambiguous rule, and both are worth raising. Sampling the highest-consequence rules first gives you evidence about the whole suite's provenance in an afternoon rather than a quarter.
  • How do you test an edge that only exists on a value the system computes internally?
    Work backwards from the derived quantity to inputs that put it exactly on its edge, one step below and one step above, and assert the derived figure rather than only the final outcome. Where the arithmetic makes that impractical through the outer interface, test the calculation at the level where the quantity is visible, and keep one case through the whole path so the wiring stays covered.
  • What do you do when two rules both claim the value sitting on a shared edge?
    Report it as a defect in the rule set, because no test can settle which rule should win — the written rules disagree. Meanwhile pin the current intended behaviour with a case that asserts the specific outcome for that exact value, so a change in evaluation order fails loudly instead of silently repricing everyone sitting on the edge.

saying these in an interview costs you the question

  • Derives edge values by reading the implementation's comparison
  • Resolves ambiguous wording privately in the test file
  • Asserts only that the edge value produced no error
  • Assumes rule evaluation order is part of the specification
  • Tests only the edges that appear on input fields
  • Blames the technique rather than the source of the edge

context