When should an LLM return Markdown instead of JSON, and why not mix both?

## Two different jobs Output format is a question about the consumer, not about taste. Markdown exists to convey structure to a *reader*: headings group, lists enumerate, bold emphasizes, and a table lays values out visually. JSON exists to convey structure to a *parser*: keys are unambiguous, values are typed, nesting is explicit, and the grammar is closed. Each is bad at the other's job. Markdown has no types, no required fields, no way to distinguish a missing value from an empty one, and no escaping story for the delimiter characters you rely on. JSON is miserable to read and worse to skim. So the decision rule is short. Output that a person reads directly — a chat answer, a drafted email, a report section, an explanation — should be Markdown. Output that a program reads — anything you will index, store in a column, branch on, or feed to another service — should be JSON or another machine format. ## Why the mixed response bites "Return the JSON and then explain your reasoning" is one of the most common first-week mistakes. The response looks fine on screen, and `json.loads()` on it raises immediately, because a JSON document is the *whole* string. The same applies to the ```json fence that models like to add: those backticks are three characters your parser has no theory for. The usual reactions are both wrong. Stripping the prose with a regex or hunting for the first `{` and the last `}` builds a fragile ad-hoc parser that breaks the first time a string value contains a brace. Asking the model more insistently not to add commentary reduces the rate but does not eliminate it, because an instruction is a bias rather than a constraint. ## The right shape: prose inside the envelope Make the machine format the outer container and put every human-facing string inside a field. ``` {"summary": "Three dishes contain peanuts.", "items": [ ... ]} ``` Now one parse gets you everything. Your code reads `items`, and your UI renders `summary` — as Markdown, if you like, since Markdown inside a JSON string is perfectly legal and just needs the usual escaping, which the serializer handles for you. This also lets you evolve: adding a `confidence` or `warnings` field later is a schema change, not a re-derivation of some text-splitting heuristic. ## Markdown as the requested format When the human *is* the consumer, ask for Markdown specifically rather than leaving it implicit, and say what structure you want: "Reply in Markdown with a level-2 heading per section and a bulleted list of findings under each." Models honour that well, and it saves you from post-processing walls of text. Be aware that Markdown itself needs care in one direction: if the model is quoting code or user content, ask it to use fenced blocks, or your renderer will interpret stray characters as formatting. ## What about Markdown tables for data? People reach for a Markdown table when the data is tabular and they want it readable. It is fine when a person reads the table. It is a poor transport format: pipes inside cell values break the row, alignment rows are noise your parser must skip, empty cells are indistinguishable from missing columns, and everything is a string. If you need both a readable table and the underlying data, generate JSON and render the table yourself — deterministic rendering is free, and unreliable parsing is not. ## A note on other machine formats JSON is the default because schema tooling, structured-output modes and validators all speak it. Alternatives have narrow niches: a single bare enum value or number is simpler as plain text when that is genuinely all you need; XML-style tags are sometimes easier for a model to keep balanced across very long outputs and are trivially extractable; CSV is compact for wide uniform rows but has the same escaping hazards as Markdown tables. Whatever you pick, the rule holds — one format per response, chosen by who reads it, with human text carried as content rather than wrapped around the payload.