In RAG, why prepend an LLM-written situating summary to each chunk?

## The orphan-chunk problem Splitting a document destroys anaphora. Inside the whole filing, "the segment reorganisation" and "the prior period" are perfectly well-defined; every chunk after the first inherits meaning from text it no longer sits next to. After splitting, that meaning is gone. The chunk is grammatically fine and semantically stranded. This hurts both retrieval channels. Dense retrieval fails because the embedding encodes a vague statement about unnamed revenue, which sits far from a query naming a specific company and quarter. Lexical retrieval fails because the terms the user typed — the entity name, the year, the section title — literally do not occur in the chunk. ## Two ways to fix it The deterministic version costs nothing: build a prefix out of metadata you already have. Document title, source system, effective date, the heading path, maybe the section number. Prepend it into the chunk text so it is embedded and lexically searchable, and store it as fields too. For structured corpora — well-titled docs with real headings — this alone closes much of the gap and it should always be your baseline. The generated version, popularised as contextual retrieval, goes further. For each chunk you prompt a small model with the surrounding document and the chunk, and ask for a short situating statement: what this passage is, where it sits, what the referring expressions in it point to. Something like "From the Q3 2025 filing of the industrial segment; this passage of the MD&A explains the 3% revenue decline against Q3 2024 following the segment reorganisation announced in June." That gets prepended to the chunk before embedding and before lexical indexing. Generated context beats metadata prefixes when the resolving information is *inside the prose* rather than in the headings — pronouns, comparatives, cross-references to earlier sections, tables whose subject was named three pages up. ## The economics The obvious objection is cost: one model call per chunk, over the entire corpus. Three things make it workable. It is an index-time cost, paid once and amortised over every query the corpus ever serves. A corpus queried thousands of times a day repays a one-off augmentation quickly. The document is the same across all of its chunks, so caching it as a prompt prefix means you pay full price for it once rather than once per chunk. That is the difference between viable and absurd for long documents. And it can use a small, cheap model. The task is descriptive, not analytical. The real costs are elsewhere: pipeline complexity, and rebuild latency. Every corpus update re-triggers augmentation for the affected documents, and a full re-chunk means re-augmenting everything. ## Pitfalls Prefix domination. If the blurb is long relative to the chunk, the embedding starts to represent the prefix. Every chunk from the same document then gets a similar vector, and you have destroyed the intra-document ranking you needed. Keep the blurb short — tens of tokens, not hundreds. Hallucinated context. The model is writing prose that will be indexed as if it were source text. Constrain it: describe the passage, do not interpret or extend it; if the document does not state something, omit it. Some teams keep the prefixed text for the index but hand the *raw* chunk to the generator, so no synthesized sentence can be quoted as fact. That is a defensible default in regulated domains. Staleness. The prefix encodes a claim about the document at augmentation time. If sections are inserted or the document is superseded, prefixes silently drift. Version the augmentation alongside the chunk. And it does not fix bad boundaries. A chunk that straddles two subjects is still confused, now with a paragraph explaining that it is confused about two things. ## Interaction with the rest of the stack The reported gains come from the combination, not the prefix alone: contextual embeddings plus a contextual lexical index plus a reranker over the merged candidates. The prefix specifically helps lexical retrieval a lot, because it injects exactly the proper nouns and dates a keyword index needs and the chunk body lacks. ## Deciding whether you need it Run the diagnostic before building the pipeline. Sample fifty chunks and read each cold: can you tell what document, entity and period it belongs to from the chunk alone? If most of them fail, you have the orphan problem and augmentation will pay. If most pass — because your corpus is well-titled and your headings are good — a deterministic prefix is the whole answer and the model calls buy you little. Then measure retrieval-failure rate on your own question set before and after, rather than importing someone else's percentages.