How should an LLM app propagate cancellation when a user closes the tab mid-stream?

## Three hops, any of which can silently fail When a user closes the tab mid-generation, the useful mental model is a chain of cancellations, each of which must be explicitly wired: 1. **Browser to your server.** The page is going away, so the request must be aborted. In practice you pass an `AbortController` signal to `fetch` and call `abort()` on unload or navigation. If you do nothing, behaviour varies by browser and by how the request was made — the connection usually dies with the page, but you should not leave it to chance, and some request modes deliberately outlive the page. 2. **Your server noticing.** The server sees a closed connection or a cancelled request context. Almost every runtime exposes this — a cancellation signal on the request, a close event on the connection — and almost none of them stop your handler for you. A handler that ignores it keeps looping over the upstream stream, writing into a socket nobody reads. 3. **Your server to the provider.** This is the hop that costs money. The provider is generating tokens until its request is closed. Aborting your own outbound call is what actually stops generation and releases the serving slot. The failure mode people miss is hop three surviving hop two: the server correctly detects the disconnect, logs it, returns from the handler — and the upstream call was made with a signal nobody cancelled, so generation runs to completion in the background. ## Why this matters beyond tidiness A long answer that nobody reads is still generated output: you are billed for the tokens produced before the abort, and every token after the user left is pure waste. Under concurrency the effect compounds — abandoned generations occupy capacity that live users are queueing for. On a page where users routinely reject the first sentence and retry, unpropagated cancellation can be a meaningful fraction of total spend. ## Decide the semantics, not just the mechanics Cancellation raises questions the plumbing does not answer: - **Do you keep the partial output?** For a chat turn, usually discard it. For a drafting tool, users are often glad to keep 300 words, so persist what arrived before aborting and mark it incomplete. Decide deliberately, because the natural implementation — abort and lose everything — is silently a product decision. - **What goes in the session log?** If the conversation history records a truncated assistant turn, the next turn is conditioned on a half-finished message. Either store it explicitly flagged as truncated, or do not store it at all; the wrong answer here produces strange follow-up behaviour that is hard to trace back. - **How does retry behave?** A retry after a partial render must replace, not append. Concatenating two independent generations produces duplicated or contradictory text. ## The client side of an aborted stream The same discipline applies when a stream dies mid-sentence for any other reason — a network drop, an upstream error at token 300. Text is already on screen and you cannot unsay it. The defensible options are to mark the fragment as incomplete with a clear affordance to continue or retry, or to roll the fragment back to the last committed boundary. What is not defensible is leaving a truncated sentence looking like a finished answer, because the user cannot tell the difference between "the model stopped here" and "the model meant this". If your renderer commits at sentence or block boundaries rather than per token, rollback becomes a clean operation rather than a guess. ## Testing it This is straightforward to verify and almost never verified. Start a long generation, kill the client, and check three things: that the upstream call ended promptly rather than at its natural completion, that your token accounting shows the truncated amount, and that no orphaned work continued behind the handler. Because nothing errors when propagation is broken — the user is gone, no alert fires, the dashboards look normal — the only way you learn about it is by looking, or eventually by the bill. ## Adjacent traps Background work kicked off by the request needs its own cancellation; aborting the model call does not stop a tool invocation or a follow-up job the handler already launched. And a keep-alive or fire-and-forget request mode that is designed to outlive the page will happily defeat hop one, so check the options you are passing rather than assuming page teardown cleans up.