skip to content

On a broker that deletes each record once it is acknowledged and stores no read position, what plays the role of lag?

level: middleimportance: nice to knowfreq 40%

answer

  1. nothing to subtract from
  2. depth replaces the count
  3. oldest waiting record replaces the age
  4. depth belongs to the queue, not a reader
  5. handed out but not acknowledged is separate

basics

~20 s

Queue depth — the count of records still waiting to be handed out — plus the age of the oldest waiting record. It is the same signal read without a position to subtract from, so it describes the whole queue rather than any one reader.

solid answer

~50 s

Where the broker stores a read position, the gap is a subtraction: newest record minus the point the reader group has reached. Where records are deleted on acknowledgement, there is nothing to subtract, because a processed record is gone rather than behind a marker. The same signal survives in a different form: **queue depth**, the number of records still waiting, and the **age of the oldest waiting record**. Both readings behave like their counterparts, including the ambiguity — a depth of ten thousand is seconds of work or a day of it depending on arrival rate. Two things change. First, depth is a whole-queue figure and cannot be attributed to a particular reader, because no record belongs to one until it is handed out. Second, such platforms usually count records already handed out but not yet acknowledged separately, and that second count is the one that distinguishes nobody picking work up from work being picked up and never finished.

go deeper

for a junior

Know that a broker which deletes records on acknowledgement still shows you how much is waiting — the queue depth — and how old the oldest waiting record is. That pair is the same signal in a different form.

for a middle

Explain why there is nothing to subtract from when records are deleted on acknowledgement, and why depth is therefore a whole-queue figure with no per-reader meaning.

for a senior

Demonstrate that you read the waiting count and the handed-out-but-unacknowledged count as two different populations, and that a depth of zero on its own proves nothing about completion.

for a principal

Consider what it means for an estate running both shapes: which readings every team must publish so that an incident on one shape can be described in terms an on-call engineer from the other can read.

## Two ways to express one idea The underlying question is always the same: *how much has arrived that has not yet been dealt with, and how long has the oldest of it been waiting?* Platforms answer it in two structurally different ways, and the difference comes down to what happens to a record after it is processed. - **Where a position is stored.** Records stay in the stream for a retention period whether or not anyone read them. Progress is a marker that advances. The gap is the distance from that marker to the newest record, and it is per reader group — two groups reading the same stream can have wildly different gaps. - **Where records are deleted on acknowledgement.** A processed record is removed. There is no marker, and nothing to measure from. What is left is simply *what is still there*: the depth of the queue, and how long its oldest occupant has been sitting. ## The readings, side by side | Question | Position-storing design | Delete-on-acknowledgement design | |---|---|---| | How much is outstanding? | Unread count from the read position | Queue depth | | How stale is the oldest outstanding work? | Age of the oldest unread record | Age of the oldest waiting record | | How far behind is *this* reader? | Answerable per reader group | Not answerable — depth belongs to the queue | | Can I re-read what was already processed? | Within the retained window, yes on designs that allow a position to be moved back | No; the record is gone | The first two rows are genuinely the same signal wearing different clothes, and the habits transfer completely: read the shape over an observation window rather than the instantaneous value, and keep the count and the age side by side because each covers the other's blind spot. ## What does not transfer Three things are different enough to be worth stating explicitly. 1. **No per-reader attribution.** A queue's depth is the work waiting for everyone. With records not bound to a consumer until they are handed out, there is no such thing as "how far behind is instance three". Diagnosing a single slow consumer has to be done from the consumer's own signals rather than from the broker's depth. 2. **A second population of records exists.** Most such designs distinguish records still waiting from records already handed out and not yet acknowledged. Those two counts answer different questions, and confusing them is the classic error: a depth falling to zero while the handed-out count climbs does not mean the work is done, it means the work has been taken and not completed. 3. **The depth is bounded by acknowledgement, not by retention.** On a position-storing platform the gap is capped by how long records are kept — beyond that, unread records are deleted and the gap silently stops growing. On a delete-on-acknowledgement platform, waiting records typically persist until someone takes them or a separate expiry removes them, so depth is more often a true count of outstanding work and less often a number pressed against a ceiling. ## Reading a depth curve Everything said about curve shapes applies here unchanged, because the model is identical — a level with an inflow and an outflow: - **Flat non-zero depth** — arrivals and completions are matched; the standing depth is in-flight bulk. - **Rising depth** — completions trail arrivals, and the slope is the difference. - **Depth at zero with a non-zero handed-out count** — consumers are taking work as fast as it arrives; the interesting number has moved to the other population. - **Depth at zero with a zero handed-out count** — either nothing is arriving, or everything is being completed immediately. As in the position-storing world, a zero is not self-explanatory and should be read next to the arrival rate. ## Why the distinction is worth knowing Most organisations run both shapes. An engineer who has only operated one tends to ask for the reading their platform provides and conclude the other is unobservable, which is wrong in both directions: neither shape can tell you how far behind one specific reader is *and* how much total work is outstanding *and* how long the oldest item has waited, without deliberately asking for each. Knowing which of those three a given platform hands you for free — and which you have to construct — is most of what operating either one asks for.

  • A queue's depth has dropped to zero but downstream work is still not appearing. What should you look at?
    The count of records already handed out and not yet acknowledged. A depth of zero only says nothing is waiting to be picked up; it says nothing about whether what was picked up was completed. If that second count is large and not falling, consumers are taking work and failing to finish it, and the records will typically be offered again rather than lost.
  • Why can you not compute a per-consumer gap on a delete-on-acknowledgement broker?
    Because no record belongs to a consumer before delivery. Work is taken, not assigned, so there is no stable mapping from a record to a reader against which a per-reader distance could be measured. The consequence is that diagnosing one slow instance has to come from that instance's own throughput and processing-time signals rather than from the broker.

saying these in an interview costs you the question

  • Says lag is unmeasurable without a stored read position
  • Reads a depth of zero as proof all work completed
  • Confuses records waiting with records handed out
  • Tries to attribute queue depth to one consumer instance
  • Assumes every broker retains records after acknowledgement