You are laying out a Redis Cluster across three availability zones. Which placement and replication decisions determine whether the cluster keeps serving — and keeps its writes — when an entire zone disappears?

## The governing constraint: majority of primaries Everything in Redis Cluster failover — escalating PFAIL to FAIL, and authorising a replica's promotion — needs agreement from more than half of the **primaries that own hash slots**. Replicas do not vote. That single sentence dictates the topology. - **Three zones, primaries spread evenly.** Three primaries (one per zone) or six (two per zone) both leave a majority after any single zone fails: 2 of 3, or 4 of 6. Note that 4 of 6 is a majority, so an even primary count across three zones still works — the danger is not evenness, it is concentration. - **Two zones cannot self-heal.** Whichever zone holds the majority is a single point of failure for the entire cluster's ability to fail over; lose it and the survivors are a minority that can neither promote nor accept writes. If only two zones exist, be honest that automatic failover across a zone loss is not achievable, and plan an operator-driven recovery (`CLUSTER FAILOVER TAKEOVER`, accepting the data-loss risk) rather than pretending otherwise. - **Never concentrate.** A layout of four primaries as 2/1/1 survives losing either small zone but not the big one (2 of 4 is not a majority). Audit placement as a property of the deployment, not a hope about the scheduler. ## Replica placement A replica in the same zone as its primary provides no protection against zone loss — the correlated failure takes both, and the shard's slots go uncovered. Enforce anti-affinity at the orchestration layer, and verify it after every rebalance: `redis-cli --cluster check` reports whether replicas are co-located with their primaries, and cluster creation tooling can be told to spread by host, but zone-awareness must generally be enforced by whatever schedules the pods or instances. Redis itself has no notion of zones. ## How many replicas One replica per primary is the minimum for automatic failover. After a zone loss, however, every promoted primary in the surviving zones is running **without** a replica, so a second failure in that window is unrecoverable and the cluster is fragile exactly when it is under stress. Options: - **Two replicas per primary** — expensive (each replica holds the full shard dataset in memory) but leaves coverage intact after one zone loss. - **Replica migration** — a primary with more replicas than `cluster-migration-barrier` (default 1) automatically donates one to an orphaned primary. A handful of spare replicas placed to survive a zone loss lets the cluster re-cover itself without operator action. This is why an "extra" replica somewhere is better than none. Also budget memory for the failure state: after a zone loss the surviving zones host the promoted primaries as well as their own, so their instances must have the headroom to be primaries (client connections, output buffers, replication backlogs) rather than quiet replicas. ## Timeouts and cross-zone latency `cluster-node-timeout` must sit comfortably above the worst realistic cross-zone round trip *and* above ordinary process stalls — fork for snapshotting, page-cache pressure, swap — or you get spurious failovers. But every extra second is also a second an isolated primary may accept writes that will be discarded. Measure the sources of pause and pick a value from data; the 15 s default is conservative and often fine, and 5 s is defensible on a low-latency inter-zone network with disciplined snapshotting. Keep the value identical on every node. Cross-zone replication also widens the asynchronous loss window: a replica two milliseconds away is two milliseconds of writes behind at best, more under load. If a data class cannot tolerate that, add `min-replicas-to-write` for that shard and `WAIT` on critical writes, and accept the availability cost. ## Partial availability policy `cluster-require-full-coverage` is the explicit choice between consistency of the cluster's advertised capability and partial service: - `yes` (default): if any hash slot has no serving primary, the cluster reports itself down and refuses queries even for healthy slots. Predictable and blunt. - `no`: keep serving the covered slots and error only on requests for the uncovered ones. For a cache tier, `no` is nearly always right — losing 1/3 of a cache is a degraded hit ratio, not an outage. For a store whose partial answers would be wrong, `yes` may be the honest setting. Decide it deliberately. ## Client-side The cluster can be perfect and the outage still long if clients do not refresh topology. Require: a client that refreshes slot maps on redirection and on connection errors, bounded socket and command timeouts consistent with the node timeout, retry policy that does not amplify load during the failover, and idempotent write shapes so a retry after an ambiguous timeout is safe. Whether the application reads from replicas is a separate decision with its own staleness consequences. ## What to test Rehearse it: kill a whole zone in a game day, measure time to `cluster_state:ok`, count lost writes with a write-and-verify harness, confirm replica migration re-covered the orphaned primaries, and check that memory headroom in the survivors was sufficient. A topology that has never been zone-failed is a hypothesis, not a design.