A regulator sets a retention floor of years on one append-only store while individual subjects may demand erasure at any time, so what design satisfies both and what does it cost to run?

## The problem, restated as a design problem You are holding one append-only store under two external rules. A **retention floor** says the history must survive for years. An **erasure obligation** says any individual subject may require that their data stops being readable, on a clock measured in days. The store's own removal machinery cannot help: it drops whole closed segments oldest-first and cannot be aimed at a subject, and dropping history early is the exact thing the floor forbids. So stop trying to change what is present and change what is **readable**. That reframing is the answer, and everything else is its consequences. ## Encrypt per subject, destroy the key The scheme has four moving parts: 1. **A key per subject.** Each subject — a person, an account, a case — is associated with its own encryption key. 2. **Encryption before the write.** The writer encrypts the payload under that subject's key and hands the store ciphertext. The store is an ordinary append-only store and knows nothing about any of this. 3. **Decryption on the read path.** Every reader that needs the payload resolves the subject's key and decrypts. 4. **Destruction as the erasure act.** An erasure obligation is satisfied by destroying that subject's key. The ciphertext is untouched, the segments still satisfy the floor, and the plaintext is gone from every place the ciphertext lives at once. The elegance is that destruction is a **single act with global reach over the ciphertext**: it does not have to visit each copy, because every copy holds bytes that were only ever meaningful through that key. The awkwardness is everything below. ## What it costs to run | Cost | What it actually looks like | |---|---| | Cardinality | One key per subject means keys scale with the population, not with the number of streams | | Read-path dependency | Something must answer "which key, and may I have it" on every read of a payload | | Irreversibility | A destroyed key is destroyed for legitimate readers too; there is no appeal | | Replay loss | The retention window is the replay budget, and erased subjects are holes in it | | Metadata leakage | Routing fields, identifiers used for placement and record sizes are usually not encrypted | Take the last two seriously, because they are what separate an answer that has been run from one that has been read. **Replay loss:** if a service rebuilds its state by replaying history, every erased subject is a gap, and the rebuild must be written to tolerate a payload it cannot decrypt rather than to fail on it. That is a property of consuming services, designed in long before the first erasure lands. **Metadata leakage:** if the thing used to place or route a record is the subject's own identifier, then after key destruction the store still proves that this subject existed, wrote at these times, at this volume. Whether that residue satisfies the obligation is a question for whoever owns the obligation, and the honest engineering answer is to state precisely what remains rather than to claim the subject is gone. ## The decision is taken before the first write This is the part that makes it a leadership question rather than a technique. Per-subject encryption cannot be retrofitted onto history: records already written in the clear are plaintext in immutable closed segments, on the store and in every copy of it. Retrofitting means rewriting an append-only history — which the store does not offer and the floor may forbid outright. So the decision has to be made when the stream is designed, for streams that may never receive an erasure obligation at all, and the cost is paid from day one against a risk that arrives later or never. That gives a small set of defensible postures: - **Encrypt per subject on the streams that carry subject data**, and accept the read-path cost there only. Usually right, and it requires knowing which streams those are — which is itself an inventory exercise most estates have never done. - **Do not carry subject payloads on the stream at all**; write a reference and keep the payload in a store that supports record-level deletion. This dissolves the problem rather than solving it, at the price of an extra lookup per read and a second system in the path. - **Accept a shorter retention window** where no floor applies, so the obligation is largely satisfied by ordinary age-based removal, with key destruction covering only the tail. Cheap, and only available where there is no floor. ## What varies between platforms Where the encryption happens differs, and the answer should not assume one shape. Where the writer encrypts before the call, the design works on any store because the store is just holding bytes. Where a platform offers to encrypt for you, what it protects is usually the medium rather than the individual payload, which does not give per-subject destruction — knowing that difference is the point. And on a platform that discards each record once it is acknowledged, the store is rarely the copy that outlives the obligation; the scheme's value there is entirely in what it does for the sinks and backups downstream. ## What an interviewer is listening for That you name key destruction quickly, then spend most of your answer on the consequences rather than on the mechanism — the pre-write decision, the new read-path dependency, the replay holes and the metadata residue. A candidate who names the technique and stops has read about it; one who says which streams they would apply it to and what they would tell the obligation's owner still remains has operated it.