Consistency & Availability

## Why the data is copied at all Every piece of data in a distributed system is deliberately stored on more than one machine: - **Durability** — a disk dies, you don't lose data. - **Availability** — one node goes down, others still answer. - **Latency** — a reader in Tokyo shouldn't have to wait on a write that committed in Virginia. The moment you have multiple copies of the same fact, you must decide how tightly those copies have to agree before you'll let a client read from any of them. That decision is "the consistency model," and it is a knob you turn per feature or per read/write path — not a single global setting for the whole system. ## Strong versus eventual, mechanically | Compared on | Strong consistency | Eventual consistency | |---|---|---| | Usually implemented by | routing writes through a single leader replica, or requiring a quorum to agree before acknowledgment | typically acknowledged locally, then propagated asynchronously | | A later read | returns that write's value or something newer | can still return the old value | | Cost | latency, and availability during a network partition | correctness: a client can observe a value that's already stale | **Strong consistency** means that once a write is acknowledged, every subsequent read — from any replica, anywhere — returns that write's value or something newer; there is no window in which a client can observe stale data. Mechanically this is usually implemented by routing writes (and often reads) through a single leader replica, or by requiring a quorum of replicas to agree before a write or read is acknowledged, so ordering is enforced before the client ever sees a result. **Eventual consistency** means that after a write is acknowledged, replicas converge to the same value over time, but a read issued moments later against a different replica can still return the old value. Writes are typically acknowledged locally and propagated asynchronously — via replication logs, gossip, or background anti-entropy — so acknowledgment is fast and doesn't wait on far-away replicas to catch up. ## Where the cost lands The trade-off exists because coordination has a real cost, and that cost lands on one side or the other. - **Strong consistency costs latency**: every write, and often every read, waits on a round trip to a leader or on agreement from a majority of replicas, which is expensive across regions and adds up under load. - **It also costs availability during a network partition** — per the CAP theorem, if a replica can't reach the leader or reach quorum, it must refuse to serve rather than risk returning stale or conflicting data, so part of the system goes dark rather than goes wrong. - **Eventual consistency buys that latency and availability back** — reads are served from the nearest local replica, writes acknowledge instantly — but the cost lands on correctness: a client can observe a value that's already stale, two different clients can get two different answers to "what is X right now," and if two replicas accept conflicting writes independently, something or someone has to reconcile them later. **PACELC** sharpens this further: even with no partition at all (the "else, latency" branch), you still trade latency against consistency, because a synchronously replicated write is inherently slower than a fire-and-forget one — the cost isn't only a partition-time cost, it's a tax paid on every single operation. ## Two recognizable failure classes In production, picking the wrong model for a given path shows up as two distinct, recognizable failure classes. 1. **Choose eventual consistency where you actually needed strong**, and you get correctness bugs that are maddening to reproduce because they depend on replica lag and routing: a payment-balance read served from a lagging replica tells a customer they have funds they've already spent; a "your order is confirmed" screen reads its own write from a different replica right after checkout and shows "order not found," a classic read-your-own-writes violation. 2. **Choose strong consistency where eventual would have been fine**, and you get latency and availability bugs instead: a "recently viewed items" widget that blocks page render on a cross-region leader round trip for no real benefit, or — worse — an entire low-stakes feature going fully unavailable during a routine network blip in the leader's region, even though nobody actually needed that read to be perfectly fresh. ## A concrete split — DynamoDB exposes both A concrete, real split: Amazon's DynamoDB exposes both modes on the very same table. - Its **default read is eventually consistent** — cheaper, lower latency, and it may reflect a slightly older replication state. - A **strongly consistent read** can be requested explicitly, at higher latency and cost, and is guaranteed to reflect the result of the most recently successful write. A checkout flow decrementing inventory before confirming a purchase would request the strong read to avoid overselling the last unit of a product; a "customers who viewed this also viewed" panel would happily use the default eventual read, because a few seconds of staleness there costs the business nothing. ## The judgment The engineering judgment isn't "always pick strong" — that tanks latency and availability everywhere, including places where nobody cares — nor "always pick eventual," which quietly ships correctness bugs into money-moving paths. It's mapping each read and write path individually to how much staleness that specific feature can actually tolerate, and paying the coordination cost only where it's load-bearing.