Why can one published record be charged for more than once on a hosted broker's bill?
answer
- one write, several dimensions
- stored, delivered, and crossing out
- copies multiply storage; subscribers multiply delivery
- re-reads bill the same bytes again
basics
~20 sA record touches several billed dimensions at once: it is counted as a billed write and its bytes, held as retained bytes multiplied by replica copies for its whole retention, then delivered to every subscriber, each delivery adding billed reads and possibly a charged boundary crossing.
solid answer
~50 sA hosted broker does not meter records, it meters dimensions, and one record passes through several of them. It arrives as a **billed write request** and as bytes written. It is then **stored**, counted as retained bytes for as long as the retention policy keeps it and multiplied by the replica copies the tier keeps on nodes. It is then **read**, and reading is billed too — as requests, as bytes delivered, and as traffic crossing a charged boundary if the reader sits outside it. Fan-out multiplies that last stage: where readers hold their own position in an append-only log, every consumer group re-reads the same bytes and each re-read bills; on queue-shaped brokers that remove a record on acknowledgement, fan-out instead means the broker holds a separate retained copy per subscription, so the multiplication lands in storage and delivery counts rather than in re-reads.
go deeper
Remember that a hosted broker meters several dimensions, not records: the same data is charged when written, while stored, and again when delivered.
Explain the two multipliers — replica copies on storage, subscribers on delivery — and why they make the bill outrun the published volume.
Diagnose a surprising bill by naming which dimension is multiplying, rather than proposing to publish less; say which shape of platform you are assuming.
Own the estate-level consequence: a shared stream with many readers turns one team's write into every other team's delivery line, and someone has to carry that cost.
## The life of one record, as bill lines On hardware you own, a record is written once to a file and read by whoever wants it; nothing is counted. On a hosted broker, the same record crosses several metered dimensions, and each crossing is a line. Walked in order: 1. **Arrival.** The write is counted as a billed request, and its bytes are counted as bytes written. A tier that meters operations counts the call; a tier that meters volume counts the payload; many meter both. 2. **Standing.** From the moment it lands it is retained bytes, and it stays retained bytes until the retention policy lets it go — multiplied by the replica copies held on nodes. 3. **Delivery.** Every read that returns it is a billed read request and a quantity of bytes delivered. 4. **Leaving.** If the reader is on the far side of a boundary the provider charges across — another failure domain, another account, the public internet — those same bytes are charged again as traffic crossing a charged boundary. So a single record can appear on four different lines of one bill, and two of those lines carry multipliers. ## The two multipliers - **Copies.** Durability is bought with stored volume. Retained bytes are charged at roughly the logical volume times the number of replica copies, so the storage line is set by a durability choice as much as by a retention choice. - **Subscribers.** Delivery is charged per delivery, not per record. Ten consumer groups reading the same stream is ten times the delivered bytes and ten times the read requests, from one write. These multiply each other in effect: a design with generous retention, an extra copy and a wide fan-out can bill an order of magnitude more than its published volume suggests. ## How the multiplication differs by broker shape This is where a candidate who knows only one platform goes wrong, because the shape decides *where* the multiplier lands. | broker shape | what fan-out costs | what a re-read costs | |---|---|---| | append-only log, readers hold their own position | one stored volume, delivered once per consumer group | a reader restarted from an older position re-reads retained bytes and bills them again | | queue-shaped, record removed on acknowledgement | a separate retained copy per subscription, delivered once each | nothing, because there is no history left to read again | | hybrid designs keeping per-subscription retention | storage scales with subscriptions, delivery with deliveries | varies by whether the subscription keeps history at all | The general statement that survives every shape: **one write is not one charge**, and the size of the gap depends on how many times the platform's model makes the data move or stand. ## Where the boundary charge lands Traffic crossing a charged boundary is the line people forget, because on their own network it was free. It is driven by where the readers are relative to the cluster, not by anything inside the cluster: - readers in the same failure domain as the cluster: usually the cheapest position; - readers in a different failure domain or a different account: charged per byte delivered, often in both directions; - a reader outside the provider's network entirely: typically the dearest byte on the bill. Because the charge is per byte delivered, it is multiplied by fan-out exactly like the delivery line is. ## Reading it back: which multiplier to attack When the bill is larger than the published volume implies, the useful question is not "how do we publish less" but "which dimension is multiplying": - if **storage** dominates, look at the copy count and at how long records stand, not at the traffic; - if **delivered bytes** dominate, count the consumer groups or subscriptions reading the same stream, and ask whether any of them are re-reading history routinely; - if **billed requests** dominate while bytes are modest, the record population is large and each one is small — the bill is counting calls, not data; - if the **boundary** line dominates, the readers' location, not the broker, is the cost driver. An engineer who can name which of those four is the multiplier has done the actual work; an engineer who reasons only from "we publish this many records a day" will be wrong by a factor they cannot explain.
- A second team starts reading the same stream from a different account — which lines move?The delivery lines: read requests and delivered bytes roughly double for that stream, because delivery is billed per delivery rather than per record. If the second account sits across a boundary the provider charges, those same bytes are billed again as traffic crossing it. Storage does not move on a log-shaped platform, because the retained bytes were already there; on a per-subscription design a new subscription can add stored volume too.
- A consumer group is restarted from an old position and re-reads a week of history. Does that show on the bill?On a platform where readers hold a position in retained history, yes: those bytes are delivered again and billed again as reads, plus any boundary crossing. It is a usage spike with no new data in it. On a queue-shaped broker that removes a record on acknowledgement there is no history to re-read, so the same operation is simply impossible rather than expensive.
saying these in an interview costs you the question
- Thinks a record is billed once, when it is published
- Assumes extra consumer groups read free because bytes are already stored
- Believes the storage line counts a record once regardless of copies
- Treats delivery to a second subscriber as something the tier does for nothing
- Ignores that readers outside the boundary add a per-byte line