Why does an attacker's encoder-tuned passage rank in first-stage similarity search despite reading badly?

### What a first stage actually computes In a retrieval pipeline over a document collection — say a literature-review assistant sitting on an open preprint archive — the index is built ahead of time. Each accepted document is cut into chunks, and each chunk is passed through an embedding model, an encoder, which returns a fixed-length vector. Those vectors are stored. At query time the user's question goes through an encoder too, **separately**, never together with any passage, and the engine returns the k chunks whose stored vectors are nearest the question vector under a distance measure. Encoding the two sides apart is what makes search over millions of chunks affordable at all, because the passage side is precomputed. The consequence that matters here is what that number contains. It is a geometric relation between two points. It has no notion of authorship, provenance, date, correctness, or whether the text reads like anything. Metadata rows beside the vector may record who submitted the document; the distance does not consult them. A chunk ranking first means it was nearest. That is the entire claim. ### The move this leaf is about An attacker who can submit into the corpus therefore has two quite different ways to be retrieved. One is to write ordinary prose that genuinely reads like the answer to the question they expect — persuading a reader and the ranking at the same time. The other, this one, is to treat the encoder as a function they can evaluate and shape text until its vector sits where a whole family of questions lands. That is optimisation against a function, not writing for a person, and the output is frequently repetitive, oddly juxtaposed, and unpleasant to skim. In the first stage that is free: nothing there opens the text. In the open-archive setting the attacker can do all of this without touching the target. The pipeline is open source, its encoder version is pinned and published, so the same function runs on the attacker's own machine. Embeddings are deterministic, so a candidate passage can be scored offline against anticipated question phrasings, iterated on for nothing, and only then submitted through the ordinary path — a preprint accepted by an archive nobody hand-curates. The interesting payoff is not one anticipated question but a geometric catch: a passage placed in a dense region can be pulled in by phrasings the attacker never enumerated. ### Why this is a narrow, brittle win The first stage is the only stage that scores the vector alone. Any later stage that reads the question and the passage together is scoring text, and text that reads badly does badly there. The fit is also to one encoder: recompute the corpus under a different model and the coordinates all move. And in a large corpus, being close is not the same as being closest — a fixed candidate budget means the passage competes with everything else near that region, most of which was written by people actually working on the subject. ### Keeping the claims pointed the right way Each of the following is a different statement and interviewers listen for the confusion: | Observation | What it establishes | What it does not | |---|---|---| | The plant tops the candidate list | its vector was nearest the query vector | that it is relevant, true or authoritative | | The plant is in the corpus at all | the submission path accepted it | that anyone read or approved it | | The assistant repeats the plant's claim | the chunk reached the model's context | that the store was compromised | None of this requires a breach. The document went in through the front door and the ranking function did exactly what it was built to do.