After a RAG multi-query fan-out returns four ranked lists, what breaks if you just concatenate them?

## What fan-out actually hands you Run four paraphrases of a terse bug-tracker title through the retriever and you get four ranked lists, each of, say, ten chunks. Naive pipelines concatenate them and pass the result to the generator. That is wrong in three separate ways, and each one has a different fix. ## Problem one: the lists overlap, and that is the point Paraphrases are deliberately close to each other, so their result sets intersect heavily. If the same chunk appears in all four lists, forty slots may hold only ten to fifteen distinct chunks. Two consequences follow. The budget consequence is obvious: you are paying tokens to send the same passage four times, and the distinct evidence you actually gained from fanning out — the chunks only one paraphrase found — gets pushed out by the repeats. The subtler consequence is that repetition in a prompt is not neutral. A passage stated four times reads as four sources agreeing, and models weight it accordingly. If the repeated chunk is a plausible-but-wrong match that every paraphrase happened to hit, fan-out has amplified an error rather than corrected it. The fix is to key on a stable identifier — chunk id, or document id plus offset — and keep one copy. Do it at the merge step. Doing it in the prompt ("ignore repeated passages") wastes tokens and trusts the model with bookkeeping; doing it after generation is too late, because the answer is already skewed. ## Problem two: id-based dedup is not content dedup Overlapping chunk windows, documents that exist in more than one source system, boilerplate headers and near-identical revisions of the same policy all produce chunks with different ids and nearly identical text. Id-based dedup lets every one of them through. Fan-out makes this worse than single-query retrieval, because different paraphrases tend to surface different near-duplicates of the same passage. If your corpus has this shape — and most enterprise corpora do — you need a second pass: collapse chunks whose text is near-identical, keeping the highest-ranked representative and, ideally, remembering that the others existed so citation still works. This is cheap to do with a hash of normalized text for exact repeats, and more expensive but more effective with a similarity threshold for near-repeats. ## Problem three: order Once you have distinct chunks, in what order do you keep them? Concatenation gives you the first list's order followed by the second's, which arbitrarily privileges whichever paraphrase ran first. The property you actually want is *agreement*: a chunk retrieved near the top by three paraphrases is a stronger candidate than one retrieved at rank one by a single paraphrase. Reciprocal rank fusion is the standard way to express that, and its practical virtue in this setting is that it consumes only rank positions. Every list came from a different query vector, so per-query similarity numbers are not a common currency you can average; positions are. That property is why RRF is reached for at the query-merge layer specifically, rather than any score-blending scheme. However it is ordered, the merged list must then be truncated to a fixed budget — the same number of chunks you would have passed without fan-out, or a modest multiple. Fan-out is meant to improve *which* chunks reach the generator, not to inflate how many do. Letting the context grow with the number of paraphrases is how teams accidentally convert a retrieval improvement into a context-dilution regression. ## Ordering the pipeline A workable order is: fan out, retrieve per query, dedup by id, collapse near-duplicates, fuse by rank, truncate to budget, and only then hand off to whatever downstream stage assembles or re-scores the context. Dedup before fusion matters: if the same chunk is still present under two ids when you fuse, it accumulates score twice and rises for the wrong reason. ## Diagnosing it in production The metric worth logging per request is the ratio of distinct chunks to total retrieved slots. If four paraphrases yield forty slots and eight distinct chunks, your paraphrases are too similar to be earning their cost — you are paying four retrievals and a rewrite call for what one query nearly gave you. If they yield thirty-eight distinct chunks, the opposite worry applies: the paraphrases have drifted so far apart that they are searching unrelated regions, and the merged context will be incoherent. A healthy fan-out sits in between, with meaningful overlap at the top of the lists and genuine novelty in the tail. That single ratio is the fastest way to tell whether a fan-out is doing anything at all, and it is worth putting on a dashboard before you start tuning the number of paraphrases.