What is the 'monotonic reads' anomaly in a replicated system, what specifically causes it, and how would you prevent a single user's session from experiencing it?

## What the guarantee says **Monotonic reads** is one of the classic 'session guarantees' that sit between full strong consistency and plain eventual consistency. Formally, it says: once a particular client has observed some data at a given version or later, that same client will never subsequently observe an earlier version of that data, no matter how many more reads it performs. It says nothing about how fresh the data is relative to real time, and nothing about what other clients see - it's purely a promise that one client's view of a piece of data only ever moves forward, never backward, across the sequence of reads that client itself performs. ## What causes a violation The concrete mechanism that causes a violation in a replicated system is **inconsistent routing across replicas with different amounts of lag**. Picture a leader with three followers: - `F1` lagging 5ms behind, - `F2` lagging 2 seconds behind because it's doing a heavy background compaction, - and `F3` caught up. A load balancer round-robins a user's successive requests across all three with no session affinity. The user's first page load is routed to F3 and shows their friend's brand-new post. They refresh a second later, and the round-robin sends this request to F2, which hasn't yet applied that post - so the post has vanished from their view, even though nothing was deleted and no data was lost; it's purely an artifact of asking a more-lagged replica the same question a moment later. This is distinct from a plain stale read, seeing old data once, precisely because it's a **regression** - the client had already seen newer data and then saw older data, which is jarring and reads to users as a bug ('it was there and now it's gone') rather than as 'data hasn't loaded yet.' ## Why it needs a name of its own Monotonic reads exists as a named guarantee because plain eventual consistency, the property that all replicas converge to the same value eventually given no further writes, says nothing about the order in which any one observer sees intermediate states along the way. Eventual consistency is compatible with a client bouncing around between different lag states arbitrarily, which is unpleasant even though the system is technically 'eventually correct.' Guaranteeing monotonic reads is a way to make eventual consistency tolerable for real users without paying for full strong consistency, which would require reading from the leader or a quorum on every request. ## The standard fix The standard fix is **session stickiness**: ensure that a given client's reads are always routed to the same replica, or more precisely to a replica whose lag never regresses relative to what that client has already seen, for the duration of their session. 1. **A simple, common implementation** is to hash the user or session ID to consistently pick one follower for that user, so the user always sees that one follower's data, which is guaranteed to move forward monotonically over time even if it's sometimes behind the absolute latest state. 2. **A more precise variant**, similar to the log-position-token approach used for read-your-writes, has the client remember the highest version or log position it has ever seen and only accept reads from a replica that has reached at least that position, falling back to a more up-to-date replica or waiting briefly if the 'sticky' replica has somehow fallen further behind, for example after a topology change or replica replacement. ## Trade-offs and failure modes The trade-offs and failure modes are subtle. | Approach | How it behaves | |---|---| | **Pure session-affinity-by-hashing** | Cheap and works well under stable topology, but breaks down if the sticky replica is replaced, for example during a rolling upgrade or replica failure - if the new replica the user gets pinned to happens to be behind where their old replica was, the client can still see a regression unless the router also checks a remembered version or watermark, not just 'same replica as last time.' It also concentrates one user's read load onto a single follower rather than truly load-balancing, which is usually fine per-user but needs care in aggregate if session assignment isn't well distributed. | | **The version-token approach** | Avoids the replica-replacement pitfall but requires plumbing a version marker through the client or session store on every request, adding complexity relative to simple hashing. | ## Where it shows up A concrete real-world scenario: a user posts a comment on a thread, sees it appear because their session is pinned to a fast follower, then their app makes a background refresh call that a naive load balancer routes to a different, more-lagged replica, and the comment appears to vanish for a few seconds before reappearing - a bug pattern reported often enough in comment-heavy apps like forums and social feeds that most production systems explicitly implement session-sticky read routing, sometimes bundled with the same infrastructure used for read-your-writes, specifically to prevent it, rather than treating it as an acceptable side effect of horizontally-scaled read replicas.