When would you not put a strict JSON schema on an OpenAI call, and what do you use instead?

## Framing the decision Strict Structured Outputs converts a probabilistic formatting problem into a deterministic one, and for most production calls that is an obvious trade. The principal-level question is where the trade stops paying, because the costs are real: a restricted schema dialect, a compile step for unseen schemas, extra output tokens for fields that must always be emitted, and coupling to one vendor's dialect. ## Case 1 — the output is prose If the deliverable is a paragraph, a draft email, or an explanation, forcing it through `{"text": "..."}` adds structure tokens, invites escaping problems, and constrains the decoder for no benefit. Use plain text. The exception is when prose travels alongside metadata the caller genuinely branches on — a confidence label, a category — and then the schema earns its place because of the metadata, not the prose. ## Case 2 — the schema is dynamic A new schema must be compiled into a grammar before decoding can start, and that compiled form is cached. Steady-state traffic against a fixed set of schemas amortises this to nothing. A product where each tenant defines their own extraction shape, or where a schema is assembled per request from user-selected fields, defeats the cache: many schemas are seen once. Mitigations before abandoning strict mode: canonicalise schema generation so semantically identical shapes serialise identically and share a cache entry; collapse a family of near-identical schemas into one superset schema with nullable fields; or warm the common shapes. ## Case 3 — the schema does not fit Two distinct limits. **Structural**: nesting depth, total property count and overall schema size are capped, and a schema that models an entire domain object graph will hit them. The fix is almost always decomposition — several narrow calls, each returning a small object, orchestrated by your code — which usually improves quality too, because a model asked for eight fields does better than one asked for eighty. **Semantic**: the dialect covers structure and a subset of type-level validation, not invariants. "The total equals the sum of the line items," "the end date follows the start date," "this identifier exists in our catalogue" — none of these are schema-expressible at any level of strictness. That is not an argument against strict mode; it is an argument that strict mode is one layer of a validation stack, never the whole of it. ## Case 4 — portability Several vendors offer schema-constrained output, and the accepted dialects and enforcement guarantees are not identical. If a call must run against more than one provider — for cost routing, redundancy, or regional requirements — you are choosing between maintaining a per-vendor schema mapping and adopting a thinner shared contract that every provider can honour, with your own validator downstream. Neither is wrong; the decision depends on how many providers, how stable the shape, and how expensive a malformed response is. What is wrong is discovering the difference during a failover. ## Case 5 — the schema is fighting the task A subtler one. Over-constraining can degrade the answer. If the schema has no way to say "absent from the source," a required field forces a fabrication; give it a nullable type or an explicit unknown member. If the conclusion field is declared before the working, the model commits to an answer and then narrates support for it, because decoding runs left to right and property order is generation order. Both are schema-design bugs that present as model-quality complaints, and both are worth naming in an interview because they show you have debugged this rather than only configured it. ## The fallback ladder 1. **Strict `json_schema`** — known, stable shape; the default. 2. **`json_object` plus a schema-shaped prompt and your own validation** — dynamic or unsupported shapes; you accept repair-and-retry as the cost of flexibility. 3. **Plain text** — prose deliverables and anything a human reads directly. Across all three, the invariant is that your own validation layer never goes away. Strict mode changes how often it fires, not whether you need it. ## Organisational angle Decide once, centrally, where response shapes are defined and how they version, so a shape change is one edit with type errors pointing at every consumer. Decide the failure policy — retry, dead-letter, human review — per outcome class rather than per call site, because refusals, truncations and validation failures want different treatment. And instrument the compile-cost tail if you are anywhere near dynamic schemas: it shows up as first-token latency, not as an error, so nothing alerts on it unless you look. ## Interview framing Answer with the ladder and one concrete cost per rung. The failure mode to avoid is dogma in either direction: "always use strict schemas" ignores dynamic and portability cases; "schemas are overkill, just parse it" ignores that repair heuristics are the thing this feature deleted.