When a reader group shares one stream's work, which membership events cause its shares to be reassigned?
answer
- membership changes, shares move
- join, clean leave, silence
- two timers, not one
- silence is inferred, not observed
- competing readers reassign nothing
basics
~20 sThree events: a member joins, a member leaves cleanly, or a member stops proving it is present — by missing its liveness signal or by holding work past its progress deadline. Designs where readers merely compete reassign nothing.
solid answer
~50 sOn platforms that split a stream into parts, each part is held by one member of the reader group at a time, and that division is recomputed whenever the system's belief about membership changes. Three things change it: a new member joining, an existing member leaving in an announced way, and a member that stops looking present — either its periodic liveness signal stops arriving, or it blows the progress deadline that bounds how long it may hold work without showing progress. The third case is the operationally interesting one, because a member can be alive, healthy and still declared gone by a timer. A clean leave is acted on immediately; a silent disappearance is only inferred once a timer expires, so its share sits unread for the whole timeout first. Queue-shaped designs where readers simply compete for records have no shares to redistribute at all.
go deeper
Recall the three triggers — a member joins, a member leaves cleanly, a member stops looking present — and that the last one covers both a dead process and a slow one.
Explain that two separate timers sit behind 'stops looking present', and why an inferred departure costs a full timeout of stalled work while an announced one costs nothing.
Show that you read a reassignment chart as a diagnosis: a rhythm nobody caused points at the progress deadline, a burst points at a deploy, and a reader vanishing is not a broker vanishing.
Frame the estate policy: announced shutdown as a deployment requirement, and a stated expectation of how often a healthy group may churn before it is treated as an incident.
## What a share is, and what actually moves A **reader group** is a set of reader processes that between them read one stream. On platforms that split a stream into parts — partitions, or a set of queues behind one name — each part is held by **at most one member at a time**, and the set of parts a member holds is its **assignment**, or **share**. A **reassignment** is the recomputation of that division. Nothing about the records changes during a reassignment. What changes is *which process is responsible for which share*, and where each new holder resumes from: the last recorded **read position** for that share, not wherever the previous holder happened to have got to in memory. Some component owns the division — the broker itself on some platforms, an elected member on others, a side table on a third — and it recomputes it whenever **the membership it believes in** changes. That phrase is the whole subject: the division follows the system's belief about who is present, not who is actually present. ## The three triggers 1. **A member joins.** A new process starts and asks to read the stream under the group's name — a deploy adding instances, an autoscaler reacting to a backlog of unread records, or an operator adding capacity by hand. Shares have to be taken from existing members and given to it. 2. **A member leaves cleanly.** The process announces that it is going before it exits — a graceful shutdown, a planned scale-in. The division can be recomputed straight away because the departure is *observed*, not inferred. 3. **A member stops proving it is present.** Most designs run two independent timers here: a short one over the periodic **liveness signal** (the process died, the host vanished, the network to it broke), and a longer **progress deadline** bounding how long a member may hold work without showing that it is still moving through it. A member that is alive, healthy and still sending liveness signals is declared gone anyway if one unit of work carries it past the progress deadline. | Trigger | How the system learns | Latency before shares move | Typical cause | |---|---|---|---| | Join | The member asks to be included | Immediate | Deploy, scale-out, manual capacity add | | Clean leave | The member announces it | Immediate | Graceful shutdown, planned scale-in | | Missed liveness signal | A short timer expires | One timeout | Crash, host loss, network partition | | Blown progress deadline | A longer timer expires | One deadline | Handler slower than the deadline allows | ## Why an announced leave is much cheaper than a silent one A clean leave and a crash both produce exactly one reassignment, but they cost very different amounts of stalled work. A silent disappearance cannot be observed — it can only be inferred from a timer — so the group carries a dead member's share, unread and unprocessed, for the whole timeout before anything moves. That is the operational argument for shutting readers down in an announced way during a deploy rather than killing them: the same number of reassignments, a fraction of the stalled time. Some designs also let a member that returns quickly reclaim the share it had, so a fast restart within a grace period causes no reassignment at all; others have no such notion and treat every restart as a leave plus a join. ## Designs where none of this happens Not every platform in this class reassigns anything. Where readers simply **compete** for records from a shared queue, no member owns a share: each reader pulls the next available record, and one that dies mid-record simply stops acknowledging it, so the record is handed to another reader once the window before redelivery expires. Adding or removing a reader on that design triggers no reassignment and no group-wide pause. Before reasoning about reassignment at all, know which of the two shapes you are on. One boundary worth keeping straight: **a member leaving is not a broker leaving**. A reader process disappearing redistributes responsibility among readers; a cluster node disappearing moves data between nodes. They look similar on a dashboard and are different incidents with different fixes. ## What an operator does with this - Expect a burst of reassignments at every deploy — that is the join and leave of each replacement instance, not a fault. - Prefer announced shutdown over process kill; it converts a timeout's worth of stalled work into nothing. - Count reassignments over time. A group that reassigns on a rhythm nobody caused is reporting something — usually a timer, not a crash. - When a group reassigns without any deploy, check the progress deadline before you check the network. The third trigger fires far more often than the first two combined in a healthy estate.
- Why does a crashed member cost more stalled work than one that shuts down in an announced way, given both cause one reassignment?An announced departure is observed, so the division is recomputed at once. A crash can only be inferred from a timer expiring, so the dead member's share stays unread for the whole timeout before anyone else picks it up. Same number of reassignments, one timeout's worth of extra backlog.
- Where does a member resume from when it is handed a share that another member was working on?From the last recorded read position for that share, not from wherever the previous holder had reached in memory. Anything the previous holder had processed but not yet recorded is handed out again to the new holder — the position store is the only handover state that survives.
- A reader group reassigns several times a minute with no deploys running. What is the first thing to check?Whether members are exceeding the progress deadline rather than dying. A handler that occasionally takes longer than the deadline gets its member declared gone while the process is alive and healthy, which looks identical to a crash on a membership chart but has a completely different fix.
saying these in an interview costs you the question
- Thinks only a crash triggers a reassignment, not a join
- Believes a member must be dead to be declared gone
- Assumes every messaging platform assigns shares to members
- Confuses a reader disappearing with a broker node disappearing
- Thinks the new holder resumes from the old holder's in-memory progress
- Treats an announced shutdown and a kill as equivalent