Restarting a reader group on the standby means either reprocessing records it already handled or skipping some — how do you choose?

## Why there is a choice at all A **reader group** restarted on a **standby cluster** cannot land exactly where it was: a **position map** gives the nearest entry at or before the group's old position, and a timestamp restart gives the first record at or after a chosen moment. So the group either begins slightly behind where it was, and re-reads, or slightly ahead, and never sees the records in between. There is no third option, and pretending the restart is exact is how a switch becomes an invisible data-loss event. The decision is therefore: **which error do I want, and how much of it**. ## Frame it as two costs, per stream | | reprocessing (land early) | skipping (land late) | |---|---|---| | what happens | some records are handled twice | some records are never handled | | visibility | usually visible downstream | silent unless something reconciles | | typical harm | duplicated side effects | missing state, missing money, missing notification | | recoverable later | often, by reconciliation | only if the records still exist somewhere | | cost to the switch | extends the restart | none — it is faster | Two things fall out of that table. First, skipping is the more dangerous default precisely because it is cheap and quiet: the group catches up instantly and nobody sees anything. Second, the cost of reprocessing lands on the handler, and whether a repeated record is harmless is a property of the consuming application, not of the broker — that is assumed knowledge here, and it is the thing you check before promising that replay is safe. ## The rule that usually holds For most streams: **land early, and land early by more than you think you need**. The uncertainty in a restart is not one record; it is however coarse your map is, plus however long the group had been running without storing its progress, plus clock differences if you used a timestamp. An overlap measured in minutes costs the switch some extra reprocessing time and removes the entire class of silent gaps. The streams that genuinely invert this rule share a shape: **only the newest record matters**. A feed of current prices, positions, or device readings gains nothing from replaying the past ten minutes, and replaying it delays the consumer from reaching the present — which is the only thing it is for. For those, restarting at the newest record on the standby is right, and the skipped records are not a loss because nobody wanted them. A third case is worth naming: streams whose consumers produce **externally visible one-way effects** — sending a message to a customer, moving money, calling a third party. Here a repeat is not merely untidy, and the honest answer is often neither replay nor gap but a deliberate pause: restart the group with its effects suppressed, let it work through the uncertain span, and re-enable them once it is past the overlap. ## How to make either choice survivable 1. **Record the choice per stream before the incident**, next to the stream's owner. During a switch you want to read the decision, not make it. 2. **Prefer a margin you can state.** "Start ten minutes early" is auditable; "start where the map says" hides how much overlap you actually took. 3. **Make the overlap measurable afterwards.** Knowing how many records were re-read turns a vague worry into a number someone can reconcile against. 4. **Never let one blanket rule cover every stream.** A single global restart policy will be wrong for the feed that only wants the present and wrong for the stream that moves money, and both will be discovered afterwards. ## Where platforms differ - Where the reader owns a rewindable position, both directions are available and the choice above is real. - On **destructive-read** designs there is nothing to rewind: acknowledged messages are gone, and what you hold is whatever the copier carried and nobody consumed. The choice collapses into a different one — whether to re-drive the work from its original source or accept the absence. - Where the records are retained only briefly on the receiving cluster, "land early" has a floor: you cannot rewind past the oldest record the standby still holds, which can quietly turn an intended replay into a gap. - Some platforms let a group's progress be set administratively while it is stopped, and others require the group to start and seek; the mechanics differ but the decision does not.