In a wiki assistant that retrieves chunks, why does approving a page's diff not mean a human read what the model reads?
answer
- two artefacts, one approval
- the reviewer opens a page, the model gets a fragment
- the diff is not the retrieval unit
- approval records a click on the file
basics
~20 sA review and a retrieval consume different artefacts. The reviewer reads an added hunk inside the whole page; the model receives one chunk cut from that page, alone. Approval evidences that a person read the file, not the fragment.
solid answer
~40 sThey consume different artefacts. The reviewer opens the page and reads the added hunk with its heading, the preceding sentence and the surrounding section on screen. The assistant never sees that object: ingestion splits the page and retrieval hands the model one chunk, which may carry the paragraph without the sentence that framed it. So an approval attests that someone found the edit unremarkable *as a page*, and says nothing about how the fragment reads alone. The change record arguably makes it worse: it puts a named approver behind text nobody read in the form the model gets. Nothing has to be hidden for this to work — both readings sit in plain sight, on different units.
go deeper
Be ready to name the artefacts: a diff hunk inside a whole page for the human, one chunk for the model. Say plainly that approval evidences a page was read, not that a fragment was.
Explain which pipeline stage produces each artefact, and why a chunk arrives in the prompt with no page-level frame around it. Expect to be asked what the split points have to do with it.
Show the judgment: state what a diff review genuinely costs an attacker and what it cannot evidence, without collapsing into either 'review is coverage' or 'review is useless'.
Own the assurance wording. If a security statement says the corpus is reviewed, be able to say exactly which claim that supports and which residual it leaves for somebody to accept.
## Two artefacts, one approval A retrieval-backed wiki assistant has a corpus anyone at the company can edit, and edits are approved by a person skimming the diff. That sounds like a review of what the assistant reads. It is not, and the gap is structural rather than a lapse of attention. There are at least four different objects in play, and interviews on this topic turn on telling them apart: | Artefact | Who consumes it | | --- | --- | | The page as rendered, whole | the reviewer | | The diff hunk, shown inside that page | the reviewer | | The chunks ingestion emits from the page | the vector store | | The one chunk retrieval returns for a query | the model | The reviewer's artefact is the richest one available: the added lines with the heading above them, the sentence before, the section around them. The model's artefact is the poorest: a fragment of a few hundred tokens, arriving in a prompt with no page-level frame, positioned as a passage that was retrieved *because it answers the question*. ## Why the split matters, not the hiding The reflex answer is that somebody sneaks unreadable content past a reader — invisible text, an odd encoding. That is a different family. Here nothing is concealed. The paragraph is fully visible and, read on the page, unremarkable: it is framed by a heading, a date, a scoping sentence, a conditional, a pronoun whose antecedent sits a line above. Those framing devices are *adjacent* to the paragraph, not inside it. Whether they travel with it depends entirely on where the split points fall — a property of the ingestion configuration, not of the text a reviewer scored. So one paragraph can carry two honest readings: archived background when read with its frame, current standing guidance when read alone. Both are real. The reviewer read one; the model was handed the other. ## What the approval record actually proves This is the direction-of-claim error worth rehearsing. An approval record proves that a person clicked approve on a diff at a time. It does not prove: - that anyone read the chunk the retriever later returned (nobody rendered it); - that every unit ingestion emits from the page reads as the page reads; - that the content is true, current, or authored in good faith; - that retrieval treats the page as more authoritative — first-stage search ranks by vector distance and has no notion of who approved what. Worse, the record is a payoff in itself. Downstream, an approved page carries an attestation: a name, a timestamp, a process. When the assistant's answer is later questioned, that record is what people reach for, and it points at an artefact that was never in the model's context. ## What it costs the attacker It is not free. The whole-page reading has to survive an attentive reader, which caps how strong the fragment can be: anything blatant enough to be reliable in isolation tends to look odd in place. The attacker also does not control the split points, and cuts move when text above the paragraph changes and the page is re-indexed — so the construction has a shelf life measured in page churn, not in model versions. Against a corpus where the retrieved unit happens to be a whole section rather than a paragraph, the two readings collapse back into one and there is nothing left. ## Where a reader usually goes wrong "We reviewed the document" is the wrong answer this question exists to correct. The model never saw a document. It saw a fragment produced by a stage the reviewer's workflow does not render. Careful reading does not close the gap either — reading the page more carefully is still reading the page, and the fragment's reading is a property of the boundary, not of attention. What careful reading does buy is attacker cost: the whole-page framing has to be plausible to somebody who is genuinely looking. ## In an interview Say the two artefacts out loud, name which stage produces each, and state what approval evidences and what it does not. Then name the obstacle honestly: the diff skim is a real control that a construction has to be authored around, not a control that has no effect. Candidates who claim review is worthless are as wrong as those who claim it is coverage.
- The change record names the approver. What does that record prove?That a person approved a diff at a point in time, and nothing more. It does not show that anyone rendered the chunk the retriever later returned, that the text is current or true, or that retrieval treats the page as authoritative. Its real effect is downstream: the attestation is what people cite when an answer is later questioned, and it points at an artefact that was never in the model's context.
- Is this any different from a page nobody reviewed at all?Yes, in both directions. The review is a genuine cost: the whole-page reading has to look unremarkable to an attentive reader, which rules out the crude plant. But the approved page ends up carrying a quality signal the unreviewed one lacks, so the same chunk arrives with a provenance story attached. The construction buys laundering, not access.
- If the reviewer read the whole page slowly instead of skimming, would that close it?No. Slower reading of the page is still reading the page; the fragment's second reading is created by where the split falls, which the reviewer's view never shows. Attention raises the attacker's cost — the framing has to be plausible rather than merely present — but it does not make the reviewed artefact and the consumed artefact the same object.
A copy-editor approves a chapter. The reader is handed one paragraph photocopied out of the middle of it. Both texts are genuine; only one of them was read.
saying these in an interview costs you the question
- We review every page edit, so the corpus is clean
- The model sees the document the reviewer approved
- Retrieval returns documents, so a whole-file review covers it
- An approved page cannot contain a directive fragment
- The approver's name proves the text was read as retrieved