How does single-token insertion or deletion let a parser continue past a missing block closer?
answer
- edit the stream, do not skip it
- insert the one token expected
- delete the one token that cannot fit
- zero-width, marked synthetic
- bounded, and progress must be real
basics
~20 sPhrase-level repair edits the token stream locally: if fabricating the one token the state expected, or dropping the current stray token, lets parsing progress, the parser does that, marks the edit synthetic, and reports once.
solid answer
~50 sBefore falling back to skipping, a parser can try the two cheapest edits at the failure point. **Insertion**: if the current state expects exactly one plausible token — typically a closer, a delimiter or a separator — the parser fabricates it with a zero-width span at the error position and continues as though the author had typed it. **Deletion**: if dropping the current token lets the parser shift the one after it, that token was stray and goes. Both are accepted only when the parser can then genuinely make progress, both are bounded so a file of noise cannot trigger thousands of guesses, and every fabricated token is marked synthetic so later phases refuse to derive further errors from it. The payoff over panic mode is that the span is still parsed, so structure comes back for it.
code
pseudocode · 18 lineson parse_error(current):
report(current, expected_set_of(current_state))
if repairs_used >= REPAIR_BUDGET: return panic_mode()
t = sole_plausible_token(expected_set_of(current_state))
if t exists:
insert synthetic token t at current.start // zero width
if parser can shift t:
repairs_used = repairs_used + 1
return RESUMED
undo insertion
if parser can shift token_after(current):
delete current // stray token
repairs_used = repairs_used + 1
return RESUMED
return panic_mode() // skip to an anchorgo deeper
Know the idea: a parser can act as though a missing closer had been typed, or ignore one stray token, so the rest of the file still yields a usable tree.
Explain both edits and the guard they share — a repair counts only if the parser can then make progress — and why the diagnostic is still reported even though parsing continued.
Show the bookkeeping: zero-width synthetic tokens, a repair budget, one repair per position, and a marker that stops later phases reporting consequences of the parser's own guess.
Decide how much guessing the front end may do on the author's behalf, and whether the same repair policy is right for a batch compile and for a file being typed.
## Repair instead of skipping Panic-mode recovery is safe but blunt: everything between the failure and the next anchor is thrown away, and no structure survives for that span. Yet most real syntax errors are far smaller than a whole construct — one token missing, or one token too many, usually because the file is mid-edit. **Phrase-level repair** (local repair) tries exactly those two edits at the failure point, and falls back to skipping only when neither works. The payoff is coverage: the construct is still parsed, so a formatter, a highlighter or a completion query still sees real structure where the mistake was. ## Insertion: fabricate the token the state wanted When a parser fails, it knows which tokens its current state could have accepted. If that set has exactly one plausible member — most often a block closer, a statement delimiter or a separator — the parser fabricates it and continues. - The fabricated token gets a **zero-width span** at the error position, so every real character in the file still belongs to a real token. - It is **marked synthetic**, so later phases can distinguish the author's text from the parser's guess. - The error is still reported. Repair changes what the parser does next; it does not make the input correct. - The repair is accepted only if the parser can then actually shift that token — otherwise it is undone and the next strategy is tried. ## Deletion: drop a token that cannot belong The mirror edit handles a stray token: a duplicated separator, a fragment left behind by an unfinished edit. If discarding the current token lets the parser shift the one after it, the token was noise, and the parser drops it with a single message instead of losing the surrounding construct. The two edits are not symmetric in risk. An insertion invents structure the author must be shown before they can trust the tree; a deletion throws away text they really did type, so it is defensible only when the token has no reading at all at that point. ## The guards that keep repair honest Unbounded repair is how a front end turns a file of noise into thousands of confident guesses. Four guards prevent that: 1. **Progress** — every repair must let the parser consume at least one further token. Without this test a repair can be re-applied at the same position forever. 2. **A budget** — a cap on repairs per file or per construct. Past it the parser drops to panic mode, which always terminates. 3. **One repair per position** — needing two edits at the same point means the parser is guessing, not repairing. 4. **Marking** — every fabricated token, and every node built over one, is flagged so later phases can refuse to derive diagnostics from the parser's own invention. ## Error productions: repair planned in the grammar Ad-hoc repair reacts to any failure. An **error production** anticipates a specific one: a rule added to the grammar that matches a mistake known to be common and, when it matches, both builds a normal node and emits a message written for that case. Because it is an ordinary rule, it fires during the parse — before any failure is raised — which is why error productions sit *above* the ad-hoc edits rather than being a fallback after them. The gain is message quality: a sentence naming the actual mistake instead of a list of expected tokens, plus a tree of the right shape. The costs are grammar weight, and the risk that a rule written to catch a mistake quietly *accepts* it if the message is ever dropped. They earn their place for the handful of mistakes common enough to name, not for the long tail. ## Which mechanism for which failure | Mechanism | What it handles | Structure recovered | Main risk | |---|---|---|---| | Error production | A specific, anticipated mistake | Full subtree plus a targeted message | Grammar weight; may silently accept the mistake | | Insertion of one token | A missing closer, delimiter or separator | Full subtree containing a synthetic token | Invents structure the author never wrote | | Deletion of one token | A stray or duplicated token | Full subtree | Discards text that may have had a meaning | | Panic-mode skip | Anything the others cannot handle | None for the skipped span | A large span with no structure at all | A practical front end layers them in that order, and each layer costs more to build than the one above it while recovering more of the file than the one below it.
- What is an error production, and when is it better than an ad-hoc repair?It is a rule written into the grammar for a mistake you expect, so the parse matches it directly and emits a message authored for that case, producing a tree of the normal shape. It beats ad-hoc repair when the mistake is common enough that a bespoke sentence helps every user — and is not worth the grammar weight otherwise.
- Why must a fabricated token be marked synthetic rather than left indistinguishable from real input?Because everything downstream reads spans. A synthetic token has zero width and no source text, so selection, formatting and navigation must not treat it as something the user typed. Worse, a phase that reports a diagnostic derived from a fabricated node is reporting the parser's own guess back to the author as their mistake.
- What stops repair from running away on a file of noise?A budget: a cap on repairs per file or per construct, a requirement that each repair advance the input, and a rule that no position takes two repairs. Once the budget is spent the parser drops to panic mode, whose skip loop is guaranteed to terminate at an anchor or at end of input.
saying these in an interview costs you the question
- Believes the repair reconstructs what the author actually intended
- Gives fabricated tokens a real width, shifting every position after them
- Applies repairs with no budget, so noisy input is repaired forever
- Treats a fabricated token as ordinary user text in later phases
- Thinks a successful repair means the error need not be reported
- Deletes a token without checking that parsing can then proceed