An erasure obligation requires one subject's records to stop being readable on an append-only stream, so why can the removal pass not carry it out?
answer
- granularity, not permission
- the pass drops whole closed segments
- oldest-first is the only order
- a floor forbids the crude fix
- unreadable, not absent
basics
~20 sThe removal pass discards whole closed segments oldest-first, never individual records, so it cannot be aimed at one subject. A retention floor usually forbids dropping that history early in any case. Erasure therefore has to make records unreadable rather than absent.
solid answer
~50 sTwo rules land on the same store: a **retention floor** saying keep this for years, and an **erasure obligation** saying one subject's data must stop being readable. They look contradictory only while you assume erasure means removing records. On a store that keeps a replayable history, records live in `segments` that are rolled closed and then never modified, and enforcement of an age bound or a byte ceiling is a background removal pass that drops whole closed segments oldest-first. It has no predicate — it cannot select a subject, and it cannot take one record while keeping the neighbour written a year later. Shortening the age bound so the records fall out is both forbidden by the floor and indiscriminate: it collapses the retention window, which is the replay budget for every reader. The operable answer is to make the subject's records unreadable, not gone.
go deeper
Recall that records on this kind of store are removed in bulk by age or bytes, not one at a time on request, and that reading a record does not remove it.
Explain the mechanics: the removal pass drops whole closed segments oldest-first, the active segment is exempt, and there is no predicate to select one subject's records.
Show the judgment that shortening the bound is both forbidden by the floor and destroys the replay budget for every reader, and move the conversation to readability as the lever.
Frame it as a decision taken before the first write: whether the estate encrypts per subject up front, and what it costs to have decided wrong once history exists.
## Two rules, one append-only store A **retention floor** is an externally imposed "keep this for at least so many years". A **retention ceiling** is its opposite, "do not keep it beyond". An **erasure obligation** is a rule or a request that one named subject's data must stop being readable. All three routinely land on the same stream, because the records a records rule wants preserved for years are the very records carrying subject identifiers. Read naively, a floor and an erasure obligation are a straight contradiction, and most candidates stop there. They are not contradictory. They only appear so while you assume that erasure means **removing records**, and removal on this kind of store is not an operation you can aim. ## Removal is coarse, and it is aimed at age On a store that keeps a replayable history, records are appended into a **segment** — a unit the store rolls closed once it is large enough or old enough, after which it is not modified again. Enforcement of an **age bound** (the maximum age a record may reach) or a **byte ceiling** (the maximum bytes a scope may occupy) is done by a background **removal pass** that drops whole closed segments, oldest-first. Three consequences follow, and all three matter here: - **The unit is the segment, not the record.** The smallest thing the store can drop is a contiguous span of history belonging to every writer at once. There is no predicate and no way to express "the records of this subject". - **The order is fixed.** Oldest-first is the only order on offer. You cannot take last year's record and keep the one before it. - **The active segment is exempt.** Whatever is still being appended to is not a candidate, so the surviving history is always somewhat more than the bound nominally allows. Reading removes nothing either. Where a store keeps history, a reader's progress is its own business and a record ten readers have read is exactly as present as one nobody read. Designs vary on this axis and it is worth saying so out loud: a queue-shaped broker that discards a record once it is acknowledged has no such history to reason about, which moves where the same obligation bites without removing it, because the copies fed from that broker still hold the payload. ## The crude workarounds, and why each fails 1. **Shorten the age bound until the records fall out.** Forbidden by the floor — dropping that history early is precisely the thing the floor exists to prevent. It is also indiscriminate: it collapses the **retention window**, which is **the replay budget** every reader depends on, to erase one subject. 2. **Rewrite the affected segments in place, minus the subject.** The store is append-only by construction. Closed segments are treated as immutable, positions are derived from their contents, and every other copy of that segment would now disagree with the rewritten one. 3. **Wait for it to age out naturally.** The floor says you may not, and an obligation with a deadline will not wait years for one. 4. **Declare the record gone because every reader has consumed it.** Consumption is not removal on a store that keeps history, and it says nothing about the copies. ## What erasure has to mean instead If the bytes must stay for the floor and the meaning must go for the obligation, the only lever left acts on **readability**, not on presence. The established technique is to encrypt each subject's payload under an encryption key belonging to that subject before it is ever written, and to satisfy an erasure obligation by **key destruction (crypto-erasure)** — destroying that key so the ciphertext still occupies the segments the floor requires but resolves to nothing. Designing and paying for that scheme is a larger question; what matters at this level is recognising that it is the shape of the answer, and that it has to be in place **before the first write**, because records already written in the clear can never be brought under it retroactively. | The rule says | What it wants | What the store can actually do | |---|---|---| | Retention floor | Keep every record for years | Hold segments and refuse to let a bound remove them | | Retention ceiling | Stop holding it after a point | Let the age bound do exactly what it already does | | Erasure obligation | One subject stops being readable | Nothing record-shaped — only make bytes meaningless | ## What an interviewer is listening for That you reach for granularity rather than for permission. The weak answer argues about whether you are allowed to delete; the strong one observes that the delete you are arguing about does not exist at that resolution, names the segment as the unit, names oldest-first as the order, and then points at readability as the only remaining lever. The second half of a strong answer is that the obligation does not stop at this store — but the first half has to be this.
- Why does the surviving history usually exceed the age bound that is configured?Because the removal pass only considers closed segments. The one still being appended to is exempt whatever its contents' age, and a closed segment is dropped as a whole, so records younger than the bound sit alongside ones older than it until their entire segment qualifies. The bound is a floor on what survives, not a precise cut.
- A subject's records were written in the clear two years ago. What can key destruction do for them?Nothing. Key destruction only makes ciphertext meaningless, and those records are not ciphertext. Retrofitting means rewriting an append-only history, which the store does not support and the floor may forbid. That is why the per-subject encryption decision has to be taken before the first write rather than when the first obligation arrives.
- Does it change the analysis if the platform discards each record once it has been acknowledged?It changes where the problem lives, not whether it exists. Such a broker holds the payload only briefly, so the store is rarely the surviving copy — but the sinks, mirrors and backups fed from it are, and the obligation follows the bytes. The erasure question becomes an inventory question sooner rather than later.
saying these in an interview costs you the question
- Just delete the subject's records from the stream
- Rewrite the closed segments in place and skip the erased rows
- Shorten the age bound until those records fall out
- Once every reader has read a record it is effectively erased
- A retention floor and an erasure obligation cannot both apply to one store
- Erasure is a permissions question, not a storage-mechanics question