In Redis Cluster, a primary that owns a range of hash slots can keep replying OK to client writes for seconds after the cluster has already promoted one of its replicas for those same slots. Explain why that window exists, what happens to those writes when the old primary rejoins the cluster, and how the `cluster-node-timeout` setting sizes that window against the risk of unnecessary failovers.

## The setup Redis Cluster shards the keyspace into 16384 hash slots, each owned by exactly one primary, with zero or more replicas per shard. Nodes exchange gossip on a dedicated cluster bus port and agree on failures there; how a node moves from PFAIL to FAIL, and how a replica campaigns and wins an election, are covered separately in this topic. What matters here is the *other* side of that story: what the losing primary is doing while all of that happens. ## Why an isolated primary keeps accepting writes A Redis primary does not ask permission before serving a write. There is no lease it must renew, no quorum round on the write path — that is exactly why a single node sustains very high write throughput. Its only self-protection is a coarse, time-based check: if a primary has been unable to reach the **majority of primaries** for longer than `cluster-node-timeout`, it puts the cluster state into `fail` locally and starts replying with errors (`CLUSTERDOWN`) instead of serving reads and writes for its slots. So the sequence during a partition looks like this: 1. The partition happens. Clients on the minority side keep sending writes to primary **P**; P applies them and replies `+OK`. It has no idea anything is wrong. 2. On the majority side, P is gossiped as PFAIL then FAIL, and after the election delay one of its replicas **R** is promoted and takes ownership of the slots with a **new, higher `configEpoch`**. 3. Independently, P's own majority-unreachable timer expires and P stops accepting writes. Steps 2 and 3 are both roughly one `cluster-node-timeout` after the partition, but they are timed by different clocks and different starting events, and the election adds its own delay while P's cutoff does not. Real deployments also produce cleaner versions of the overlap: a primary that was **stalled** (a long fork/copy-on-write pause, swap, a hypervisor freeze) is failed over while unresponsive, then wakes up, briefly serves the clients still pointed at it, and only then discovers it is out of date. Clients with a cached topology map keep it company — a client only refreshes on a `MOVED` redirect or an error, and a stranded primary is issuing neither. The important framing for an interview: the exposure is **bounded by `cluster-node-timeout`, not by network round-trip time**. With the 15000 ms default, that is potentially fifteen seconds of writes that were acknowledged and are already doomed. ## What happens on rejoin: configEpoch decides Every primary carries a `configEpoch`, a monotonically increasing number that acts as the version of its slot-ownership claim. When R won its election it took a `configEpoch` strictly greater than P's. Slot-ownership conflicts in Redis Cluster are resolved by **the greater configEpoch wins** — this is the whole conflict-resolution rule, and it is why the cluster never ends up with two permanent histories. When the partition heals, P receives gossip and heartbeat packets advertising that R serves P's slots at a higher epoch. P does not argue and does not merge. It updates its slot map, **reconfigures itself as a replica of R**, and synchronizes. Because P wrote past the offset R was promoted at, its extra data cannot be reconciled by a partial resynchronization: it loads R's dataset and its own divergent writes are gone. No client is told; the application simply finds, later, that a write it saw succeed never existed. A manual `CLUSTER FAILOVER` is the deliberate contrast: the primary pauses clients, waits for the replica to match its offset, and hands over — a coordinated switch with no divergence. That is the tool for planned maintenance; the window described above is what an *unplanned* failover costs. ## Sizing cluster-node-timeout `cluster-node-timeout` (default 15000 ms) simultaneously controls: how long a node must be unreachable before peers suspect it, how long an isolated primary keeps writing, and — via `cluster-replica-validity-factor` — how stale a replica may be and still be electable. Lowering it directly shrinks the divergence window and shortens outages. The cost is **false failovers**: any stall longer than the timeout looks identical to death. Real sources of such stalls are ordinary — fork and copy-on-write pauses during `BGSAVE`/AOF rewrite on a large dataset, page-cache and swap pressure, noisy-neighbour or cross-zone network jitter, a slow `KEYS`-shaped command blocking the single-threaded event loop. Each false failover is not free: it triggers a promotion, a resync, a client topology refresh, and its own window of discarded writes. Setting it to a few hundred milliseconds converts a rare correctness problem into a frequent availability problem. The engineering answer is to measure: take the p99.9 of observed process pauses and inter-node round-trip time, and set the timeout comfortably above that — commonly 5000–10000 ms for a single-region cluster, higher when nodes span zones with jittery links. Then treat the residual window as real and design around it: keep money-like data in a store that acknowledges after durable quorum commit, make Redis-held state reconstructible, and make client retries idempotent (write an absolute value rather than `INCR`) so a retry after an ambiguous failure is safe.