Where must a deletion request reach in an LLM app beyond the app database?

## Why this is harder than in a normal service In a conventional application, a customer's data lives in a handful of tables plus backups, and deletion is a well-understood cascade. An LLM application copies the same text into a set of stores that were built for debugging, evaluation and performance, each added by a different team at a different time, and several of them run outside your infrastructure. A deletion routine that only touches the transactional database is, in this architecture, close to a no-op. The honest way to answer the question is to trace one request end to end and name each place its content came to rest. ## The inventory **Trace and log stores.** Prompts and completions are captured for debugging, cost attribution and quality review. This is usually the single largest copy of the sensitive text in the system, frequently held by a third-party observability vendor, with retention set by whoever configured the integration and read access granted broadly to engineering. If deletion reaches nothing else, it must reach here. **Eval and golden datasets.** Mature teams mine production traces for eval cases — that is the recommended practice, and it is also a data-propagation path. A trace sampled into a golden set stops being an ephemeral record and becomes a fixture: committed to a repository, copied into CI artefacts, snapshotted alongside experiment results. Removing the file from the working tree does not remove it from git history, and re-writing history on a shared repository is a heavy operation. The mitigation is upstream: fixtures are built from redacted text, or synthesised from real traces rather than copied verbatim. **Agent memory.** If the product remembers users across sessions, there are two layers: an episodic log of what happened, and semantic facts extracted from it. A deletion must clear both, and the extracted facts are the harder half — a fact derived from a conversation no longer contains the conversation, so it is only findable if the write path recorded provenance. This is the practical argument for storing, with every memory record, the subject and source it came from. **Vector-index payloads.** Retrieval systems almost always store the source chunk next to the embedding, because you need the text to put in the context. Deleting the row deletes both. If for some reason only the vector remained, note that embeddings are not anonymised data — inversion research shows meaningful reconstruction of source text from embeddings — so they are in scope too. **Cache tiers.** A semantic cache keyed on request similarity holds request and response bodies. Provider-side prompt caches hold prefixes for a TTL. Neither is addressed by a database delete; the first you must purge, the second expires on its own schedule and you can only account for it. **Provider retention.** Whatever the provider's bounded window is, your deletion request does not shorten it. What you can do is know the number, state it to your reviewers, and ensure the content sitting inside that window was tokenized in the first place — which is exactly why the redaction boundary and the deletion story are the same design problem. **Fine-tuning corpora and weights.** The uploaded corpus is a deletable object. The model trained on it is not: there is no reliable mechanism to remove specific records from weights, so the remedies are coarse — retire the model, or retrain from a corrected dataset. Because this copy is uniquely irreversible, fine-tuning inputs deserve the strictest redaction of anything in the pipeline. **Backups and warehouses.** Every store above has backups, and several are replicated into an analytics warehouse. Backup-cascade policy is a normal engineering problem, but it is now a problem in six systems instead of one. ## Making it tractable Deletion is only cheap if it was designed in. Three moves make the difference: 1. **A stable subject identifier propagated everywhere.** Every trace, memory record, cache entry and eval fixture carries the identifier of the person the content concerns. Deletion becomes a query per store instead of a text search. 2. **Redaction upstream of the fan-out.** If the text that reaches traces, evals and caches is already tokenized, deleting the vault entry breaks the link for every downstream copy at once. This does not discharge a deletion obligation on its own, but it collapses the blast radius from "seven stores hold plaintext" to "seven stores hold surrogates and the map is gone." 3. **A written data-flow inventory, maintained.** New sinks appear constantly — a new tracing vendor, a caching layer, an experiment store. The inventory is only useful if adding a sink is a step that updates it. ## The shadow copy The classic incident is not architectural. Someone enables full prompt capture behind a debug flag to chase a bug, the flag stays on for two weeks, and a second complete copy of every client note lands in a log store nobody included in the review. When the deletion request arrives, the inventory is silently wrong. Treat verbose prompt logging as a change that requires the same review as adding a database. ## What interviewers listen for That you enumerate stores rather than gesture at "we delete the data"; that you know evals and agent memory are propagation paths; that you say plainly which copies cannot be deleted and what you do instead; and that you make the whole thing tractable with a propagated subject identifier and upstream redaction.