skip to content

When a parser recovers from an unexpected token in panic mode, how does it choose where to resume?

level: middleimportance: must knowfreq 60%

answer

  1. report once, then stop listening
  2. skip to a token you can trust
  3. anchors: delimiters, closers, declaration keywords
  4. unwind rules until one accepts it
  5. consume the delimiter, leave the closer

basics

~20 s

Panic mode discards tokens until it reaches one from a synchronising set — a statement delimiter, block closer or declaration keyword — then unwinds its suspended rules until one accepts that token, and parses on.

solid answer

~50 s

Panic-mode recovery has two halves. First the parser reports the failure **once** and then discards input without trying to understand it, until the current token belongs to a **synchronising set** chosen for the construct being parsed — commonly a statement delimiter, a block closer, or a keyword that can only begin a fresh declaration. Second it unwinds its own state: returning from suspended recursive-descent calls, or popping parser stack states, until some rule can legally continue on that token. A statement delimiter is normally consumed, because it ends the broken construct; a block closer is normally left in place, because the enclosing rule is the one that needs it. The aim is not to guess what the author meant, but to reach a point where the rest of the file is parsed on its own merits, so a genuinely separate later mistake is still reported.

code

pseudocode · 15 lines
pseudocode
on parse_error(bad_token):
    report(bad_token, expected_set_of(current_state))
    suppress_reports = true

    while current is not in SYNC_SET and current is not END_OF_INPUT:
        advance()                       // discarded, never reported

    unwind rules until some rule can continue on current

    if current is a statement delimiter:
        advance()                       // consume it: the statement is over
    // a block closer or declaration keyword is LEFT for the enclosing rule

    suppress_reports = false
    resume()

go deeper

for a junior

Know that a real parser does not stop at the first mistake: it reports it, skips forward to a token that clearly starts something new, and carries on parsing from there.

for a middle

Explain both halves — discarding input up to an anchor and unwinding the parser's own suspended rules — and say why a statement delimiter is consumed while a block closer is left for the enclosing rule.

for a senior

Show how you would pick anchors for a real grammar, keep a stack of them for nested constructs, and stop recovery resuming so early that the same mistake is reported twice.

for a principal

Weigh recovery quality against front-end complexity: how much of an error list your users can trust, and whether one recovery policy can serve both a batch compile and an interactive editor.

## The problem panic mode solves A parser detects an error at the first token it cannot use. At that instant it knows two things: the input is wrong, and its own stack of half-finished rules describes a shape the text does not have. Stopping there is tolerable for a batch compile of a small file. It is not tolerable for a background syntax service in an editor, where the file is nearly always incomplete and the user still expects structure for the parts that are fine. **Panic-mode recovery** is the cheapest way back to solid ground: report once, throw input away until the input itself says a new construct is starting, and unwind the parser until some suspended rule can accept that token. ## Choosing the synchronising set The set of tokens the parser will stop skipping at is the whole design. A good synchronising token is one whose appearance is strong evidence about structure regardless of what came before it: - **Statement delimiters** — the token that ends one statement says the next statement begins after it. - **Block closers** — a closing bracket or block-ending keyword says the enclosing construct is over. - **Declaration keywords** — tokens that can only begin a top-level or member declaration. - **End of input** — the stop of last resort, which is what makes skipping guaranteed to terminate. The formal version of the same idea is to synchronise on the **FOLLOW set** of the rule that failed: those are exactly the tokens that can legally appear after that construct, so seeing one means the construct is finished. Hand-written recovery usually takes a smaller, hand-picked subset, because FOLLOW sets are computed across the whole grammar and contain tokens far too common to be trusted as anchors. ## Unwinding the parser's own state Skipping input is only half the job — the parser's state has to be unwound to match. The sequence is: 1. Report the error once, at the offending token, with the expected set for the current state. 2. Discard tokens until the current token is in the synchronising set, or is end of input. Discarded tokens produce no messages. 3. Unwind suspended rules — return from recursive-descent calls, or pop parser stack states — until a rule remains that can legally continue on the current token. 4. Decide whether to consume that token or leave it for the rule that will use it. 5. Resume normal parsing and re-enable reporting. Step 4 is the one that is most often wrong. A statement delimiter is normally **consumed**: it closes the broken statement, and leaving it hands the next rule a token it also cannot start with, producing a second message in the same place. A block closer is normally **left**, because the enclosing rule needs it; consuming it swallows the end of the block and makes everything after it look nested one level too deep. | Synchronising token | What it signals | Consume or leave | Typical failure if you get it wrong | |---|---|---|---| | Statement delimiter | The broken statement is over | Consume | An immediate second error on the same token | | Block closer | The enclosing construct is over | Leave | The rest of the file parses as if still inside the block | | Declaration keyword | A new member or top-level item begins | Leave | The declaration header is swallowed and its body parsed loose | | End of input | There is nothing to recover into | Leave | An endless skip loop if the loop forgets this case | ## Why a crude strategy survives Panic mode makes no attempt to reconstruct the author's intention, and that is exactly why it is robust. Provided each recovery round advances the input by at least one token, it terminates. It needs no per-rule repair logic, so it scales uniformly over a large grammar. And because it lands on a real construct boundary, the errors reported after resynchronising are usually **independent** of the first one — the property that makes a list of errors worth reading at all. The cost is precision. Everything between the failure and the anchor is unparsed, so no structure exists for that span, and one stray opening bracket can cost a whole block. That is acceptable for a batch compile and often unacceptable for an editor, which is why real front ends layer finer strategies on top: a single-token insertion or deletion attempted first, an error node recorded so a partial tree still covers the skipped range, and panic mode kept as the fallback that always works. ## Symptoms of a badly chosen anchor set - Anchors that are too common: recovery stops before the broken construct is really over and reports the same mistake again. - Anchors that are too rare: a small typo costs hundreds of lines of structure. - A single flat anchor set for a nested grammar: an inner rule consumes a token that belonged to an outer one, and nesting is wrong for the remainder of the file.

  • Why is panic-mode recovery guaranteed to terminate, and what must a recovery routine do to keep that guarantee?
    Because the skip loop is monotone in input position: it either advances past a token or stops at end of input. The guarantee only holds if every recovery round consumes at least one token. A routine that can resume without consuming anything must force one advance before retrying, otherwise the same token re-enters recovery indefinitely.
  • How do nested constructs change which synchronising set a recovery routine should use?
    Each active rule contributes its own anchors, so practical recovery keeps a stack of synchronising sets and stops at the innermost match. A token that closes an outer construct should unwind several rules at once rather than being consumed by the innermost one, which is why recovery pops frames until it finds a rule that can genuinely continue.
  • What is permanently lost for the span between the failure and the anchor?
    All of it: those tokens are never parsed, so no nodes exist for them and no later phase can say anything about that range. Front ends that serve editors compensate by recording an error node covering the skipped span, so the tree still spans the whole file and positions after it remain correct.

A reader who loses their place in a long list of instructions does not guess which step went missing; they scan forward to the next numbered heading and start again cleanly from there.

saying these in an interview costs you the question

  • Thinks the parser guesses what the author intended to write there
  • Emits a syntax error for every token it skips
  • Consumes a block closer during recovery, nesting the rest of the file wrongly
  • Believes recovery restores structure for the skipped span
  • Uses one very common token as the sole anchor, so recovery stops too early
  • Forgets end of input as a stopping condition for the skip loop