In an ordered-choice grammar for inline text annotations, `@notebook` parses as `@note` followed by the letters `book` — why?

## The defect The grammar looks reasonable on the page: ``` Annotation <- "@" Name Name <- "note" / "notebook" ``` At the position after `@` in `@notebook`, the choice tries `"note"` first. It succeeds, consuming four characters, and the choice returns that result. The alternative `"notebook"` is **never attempted at that position** — not because it would fail, but because nothing asks it to. `Annotation` then completes, and the caller resumes at `book`, which the surrounding text rule happily swallows. The document parses; it just parses into the wrong shape, which is the worst failure mode a grammar has. Two properties combine to produce it: - **First success wins.** Ordered choice compares position in the list, never how much input an alternative would consume. There is no longest-match rule anywhere in the formalism. - **A succeeded expression is committed.** Even when the enclosing rule later fails, the failure propagates outward to the nearest enclosing choice rather than back into the one that already returned. ## Why no earlier stage saved you In a design where a separate stage first groups characters into units, that stage would have produced one unit for the whole word before the grammar ever saw it, and a literal comparison against `note` would simply not match. A **scannerless** parser has no such stage: its rules match raw characters, and the only grouping that exists is the grouping the rules perform. So every place where one name is a prefix of another is a place where the grammar author, not the machine, has to decide what a boundary means. ## The two fixes, and what each one is for ``` Name <- "notebook" / "note" (1) longer alternative first Name <- ("notebook" / "note") !Letter (2) also require a boundary ``` 1. **Ordering** handles names that are both in the list. With `"notebook"` written first, `@notebook` matches the long name and `@note` still matches the short one, because the long alternative fails on that input and the choice falls through. 2. **The not-predicate** handles names that are *not* in the list. `!Letter` succeeds only when the next character is not a letter, and consumes nothing. On `@notepad`, `"notebook"` fails, `"note"` succeeds, and `!Letter` then fails at `p`, so the whole annotation is rejected — a loud error instead of a silent truncation. The order of these two fixes matters: the predicate alone does **not** repair the bug. With `("note" / "notebook") !Letter`, the choice still commits to `"note"` on `@notebook`, the predicate fails at `b`, and because a committed choice is not re-entered, the rule fails on a perfectly valid annotation. The predicate makes the grammar stricter; only the ordering makes the long name reachable. ## A third option: stop letting the choice decide how much to consume ``` Name <- Letter+ ``` One rule consumes the whole word, and a later step checks which annotation the text names. The hazard disappears because no choice is deciding a length any more. | Approach | Unknown name behaves as | Grows well with many names | |---|---|---| | Longest-first choice | parse error at the annotation | ordering must be maintained by hand | | Choice plus `!Letter` | parse error, reported at the name | same ordering duty, stricter boundary | | Generic name plus later check | parses, then fails a semantic check | new names need no grammar edit | The third row is usually what a maintained syntax settles on, because it also gives a better message: the parser can say *which* name it did not recognise, rather than pointing at the character where the last alternative died. ## What the symptom looks like in practice - A name that is a strict prefix of another silently wins, and the remainder shows up as stray text in the output. - Adding a new name to the end of a choice list appears to do nothing, because an earlier alternative already covers its prefix. - The reported error position is far from the real cause, since failure surfaces wherever the truncated remainder finally breaks a rule. - Reordering the list for tidiness changes behaviour, which is why alternative order belongs in review rather than in a formatting pass. The habit worth acquiring: whenever alternatives in one choice share a prefix, decide explicitly what should happen at the boundary, and encode that decision with ordering plus a predicate — the grammar will not raise the question for you.