A stream ingests 400 GB a day, is kept for seven days, and the cluster keeps three copies - what stored-byte figure should planning use?

## The multiplier, stated once Everything on this subject follows from one line: > **stored bytes = ingest rate x retention window x copy count** The middle and right factors together are worth naming: **the storage multiplier** is `copy count x retention window`, the factor that converts a rate into a footprint. In the worked case it is `3 x 7 days = 21 days of ingest`, which is why a stream taking in 400 GB a day needs roughly 8.4 TB of stream data on disk. Because the total is a product, each factor scales the whole thing. Halving the retention window and dropping one copy of three are not additive savings: the first removes half the total and the second removes a third of what remains. And because it is a product, a small factor on a large base is worth more than a large factor on a small one - which is why the first question about any storage bill is *which stream* rather than *which setting*. ## Working a figure end to end 1. **Take the rate at the peak, not the average.** Disks are sized by the busiest sustained period, because a volume that fills is an incident and not a line on a report. 2. **Multiply by the retention window** actually in force on that stream, including any per-stream override rather than the cluster default somebody assumes is in force. 3. **Multiply by the copy count** for that stream. Again, the effective one, which a per-stream override may have changed. 4. **Add per-record and index overhead.** Records carry headers and keys, and the on-disk layout carries its own structures; the raw payload total understates the footprint. 5. **Add headroom for a rebuild.** When a node is replaced, a copy is rebuilt from scratch, and the cluster must have somewhere to put it while the old one may still exist. 6. **Add ordinary operating headroom** so that a slow reader, a stuck deletion or a retention change does not run the volume to the edge. A common shape is to provision around a third above the raw product - `8,400 GB x 1.3 is about 10.9 TB` - but the number is a judgment about your own failure modes, not a constant to copy. ## When there is no retention window This is the direction the arithmetic most often gets asserted wrongly. On platforms where a stream is a retained log, records survive for a window regardless of whether anyone read them, and the window is the multiplicand. On platforms where a record is **deleted once it has been acknowledged**, there is no such window: what is stored is only what has not yet been consumed, so the multiplicand is **backlog depth (stored undelivered bytes)**. That difference changes the character of the planning problem completely: - With a retention window, the footprint is **predictable and bounded** - it is a rate times a window you chose, and it does not care what readers do. - With backlog depth, the footprint is **small in the normal case and unbounded in the bad one**. Plan it against the depth you are willing to survive - how long readers may be down before the volume is in trouble - rather than against normal-day depth, which tells you almost nothing. ## Where the arithmetic differs by design | Design | Multiplicand | Copy factor | |---|---|---| | Retained log, leader plus followers | rate x retention window | the copy count you set, per stream | | Delete on acknowledgement | backlog depth (stored undelivered bytes) | the copy count, where the design has one | | Exactly one paired copy | rate x window, or backlog depth | fixed at two; not a lever | | Shared durable storage beneath the brokers | rate x retention window | none at the broker; the store keeps its own | The point of the table is not to memorise four cases. It is that a candidate who states the multiplier as though every broker were the one they know is giving an answer that is wrong for half the room. ## Reading the result The figure is useful in three ways beyond sizing a volume. It tells you **what a change is worth** before you make it - one day off a seven-day window on this stream frees about 1.2 TB, a copy off three frees about 2.8 TB. It tells you **where the estate's bytes actually are**, which is usually a couple of streams and not the long tail. And it tells you **what you are committing to when you approve a posture**, which is the honest reason to run the multiplication in a design review rather than after the bill arrives. What the figure deliberately does not contain is any rate: how bytes stored and bytes moved are priced, in what unit and across which boundary, is a platform-billing subject. This arithmetic produces the quantity, and the quantity is the part engineers are expected to get right.