A HyDE draft invents a false drug dosage — does that break retrieval, and when does invention actually hurt?

## Two very different kinds of wrong When people hear that HyDE searches with a model-generated passage, the immediate objection is: the model hallucinates, so the search will be wrong. The honest answer is that hallucination in a HyDE draft splits into two cases with completely different consequences. **Case one: wrong number, right neighbourhood.** Suppose a parent on a rare-disease forum asks about a topical treatment for their child's blistering skin, and the drafted passage confidently states a specific milligram-per-kilogram dose that is simply invented. The draft still reads like a paediatric dermatology passage: it uses the disease name, the anatomical vocabulary, the treatment class. When the encoder compresses it to a vector, the invented number contributes almost nothing to the direction of that vector, while the terminology contributes almost everything. The search lands among genuine dermatology passages. The draft is then discarded, and the real retrieved documents — which carry the real dose — are what the generation step sees. Retrieval was not harmed. **Case two: domain drift.** Now suppose the query is terse and ambiguous, and the model resolves it the wrong way — drafting a passage about a cardiac condition when the question was dermatological. The draft is internally coherent and fluent, but it is *about something else*. Its embedding sits in a different region of the space, and the search returns passages that are highly similar to the probe and entirely irrelevant to the user. This is the failure mode that matters. ## Why drift fails silently A lexical search that goes wrong tends to return nothing, which is a loud signal. A dense search with a drifted probe returns a full page of confidently-scored results. The similarity scores look healthy because the probe and the retrieved passages genuinely are similar — to each other. Nothing downstream can tell that the probe was about the wrong subject. If the generation step is instructed to answer from context, it will produce a fluent answer grounded in irrelevant documents, which is worse than an obvious empty result. ## The acronym-collision trigger Drift is most commonly triggered when the query contains a term whose sense depends on the corpus, and the model must guess that sense with no corpus evidence available. Acronyms are the worst offenders because the same three letters carry unrelated meanings across industries, and the drafting model picks whichever sense dominated its training data rather than whichever sense your index uses. Ordinary polysemous words behave the same way: a single word can pull an entire drafted paragraph into the wrong professional register. Once the draft commits to a sense, the whole vector commits with it — expansion amplifies the error instead of hedging it. ## Detecting it Because the symptom is confident wrongness, you have to look for it deliberately: - **Result-set overlap.** Run the same query with and without HyDE and measure how much the top-k sets intersect. Near-zero overlap on a query where the plain query already worked is a drift alarm, not a win. - **Offline recall on a labelled set.** Keep a held-out set of queries with known relevant documents and track recall with HyDE on and off. Report it per segment, because HyDE can lift recall overall while destroying it on the ambiguous-acronym slice. - **Score distribution is not a signal.** Do not use similarity scores as a drift detector; drifted probes score well by construction. ## Mitigating it Several cheap guards exist, and they compose: - **Blend with the original query.** Average the draft's embedding with the raw query's embedding, or concatenate the query text into the drafted passage before embedding. The user's own words act as an anchor that limits how far a drifting draft can pull the probe. - **Average several drafts.** Sampling a few hypothetical passages and averaging their vectors makes a single bad draft a minority vote rather than the whole probe. - **Keep drafts short.** A one-paragraph draft has less room to wander into a second subject than a long one. - **Constrain the prompt with domain context.** Telling the drafting model what kind of corpus it is writing for — a clinical archive, an HR policy set, an incident-response knowledge base — collapses most acronym ambiguity before it starts. - **Fall back.** If the HyDE-retrieved set has near-zero overlap with the plain-query set, retrieving with both and taking the union is a reasonable safety net for high-stakes domains. ## The framing an interviewer wants The strong answer is not "hallucination is fine" or "hallucination breaks it" — it is the distinction. Factual errors inside the right topic are tolerated by the geometry; topical errors are not, and they fail quietly. That distinction also explains why HyDE is safer than it sounds in specialist domains where the model knows the register but not the specifics, and riskier than it sounds in domains full of internal acronyms.