What is the difference between JSON mode, a strict schema mode, and grammar-constrained decoding?

## The question behind the question An interviewer asking this wants to know whether you can tell *syntax* from *schema* from *language*. Teams routinely ship a pipeline on a JSON mode, see clean parses in testing, and then discover in production that a third of the rows are missing a field the downstream loader requires. The three tiers give strictly increasing guarantees at strictly increasing cost and rigidity. ## Tier 0 — a prompt instruction only Asking for JSON in the prompt gives you no guarantee at all. The model may wrap the object in prose, open with a markdown fence, or trail an explanation. This tier is a probability, not a contract, and it degrades exactly on the inputs that are hardest — long, ambiguous, or unusual documents. ## Tier 1 — JSON mode: syntax only A JSON mode forces the output to be a parseable JSON value. Under the hood this is already a constraint on the decoder: at each step only tokens that can continue a legal JSON document are allowed, so unbalanced braces, trailing commas, and unquoted keys become impossible. What it does **not** know is your schema. Extracting reaction yield, solvent and temperature from an organic-chemistry abstract, a JSON-mode response can legitimately be `{}`, `{"note": "not reported"}`, or `{"yield": "about 70%"}` — all valid JSON, none loadable into a fixed record. This is why JSON mode is described as legacy: it removes the parse error and leaves the schema error. ## Tier 2 — strict schema mode: conformance Here you supply a schema and the provider compiles it into the decoding constraint. Required keys must appear, a numeric field cannot receive a quoted string, an enum field can only hold a listed member, and unknown keys cannot be opened when the schema forbids them. The practical consequence is that the *shape* of the output stops being a source of production incidents; your failures move from parsing to semantics. The price is expressiveness. Strict modes typically support a subset of JSON Schema, and the limits are worth knowing because they are where deployments trip: - every property may have to be listed as required (optionality expressed as a nullable type instead), - open-ended objects and unconstrained additional properties are often disallowed, - nesting depth, total property count, and schema size are bounded, - arbitrary composition (deeply nested unions, conditional subschemas) is frequently unsupported, and - string `pattern` support varies, so a regex constraint is not universally available. A schema that models a union of eight variants with conditional branches usually has to be flattened — one discriminator enum plus a wide flat record — before it can be enforced at all. ## Tier 3 — a full grammar: any formal language A grammar constraint drops the assumption that the target is JSON. You give a context-free grammar and the decoder is held to it, which lets you generate a restricted SQL subset, a configuration DSL, a strict CSV line, or a domain notation. It also lets you express things a schema cannot: interleaved literal text, an ordering requirement, or a token-level format for a field. The cost is that you now own a grammar — an ambiguous or over-permissive one produces surprising output, and an over-tight one can leave the decoder no legal continuation at some state. ## What none of the tiers gives you All three constrain *form*. None constrains meaning. A schema-valid record can carry a solvent that the abstract never mentions, a yield of 250 percent, or a temperature copied from a different reaction. Range checks, cross-field consistency, referential checks against your own tables, and plausibility rules belong in a validation layer above the decoder, and that layer is not optional at any tier. ## How to choose Default to a strict schema mode with a flat, enum-heavy schema and business-rule validation on top. Drop to a grammar only when the target is not JSON or when you need a token-level format the schema dialect cannot express. Reach for a bare JSON mode essentially never — its only remaining role is compatibility with an endpoint that offers nothing stronger.