In an asynchronous primary-replica setup, what causes replication lag, and what techniques can an application use to avoid serving a user stale data right after their own write?

## What lag actually is Replication lag is simply the gap in time between a write being durably committed on the primary and that same write becoming visible on a given replica. In an asynchronous replication setup - the default for most relational and NoSQL systems because it keeps write latency low - the primary acknowledges a write to the client as soon as it is durable locally, then ships the corresponding log entry (a WAL segment in Postgres, a binlog event in MySQL) to replicas in the background. The replica reads that stream and re-applies each change in order. ## What widens the gap Under normal conditions this happens in tens of milliseconds, but several things can widen the gap: - the primary accepting writes faster than a replica can apply them (a burst of traffic, a bulk import, or a single very large transaction the replica must apply as one unit); - network latency or bandwidth constraints, especially across regions; - the replica itself being busy doing something else, like serving a flood of read queries or running index/vacuum maintenance; - or the replica simply running on weaker hardware. Lag can range from milliseconds to, in pathological cases, minutes. ## Why it matters to a reader This matters operationally because many architectures route reads to replicas specifically to spread load off the primary. If lag is nonzero, a client that just wrote data and then reads it back from a replica can see the pre-write state - the classic **'read-your-writes' violation**, one of the most common bugs reported in systems that naively load-balance reads across replicas. A subtler problem is **monotonic-read violation**: if a client's successive reads are load-balanced across replicas with different lag, it can see data 'go backward in time' - read a value, then on the next request read an older value from a less-caught-up replica. A third problem is **causal consistency** across related writes: if write A causally precedes write B (e.g. a comment posted after a post), and B propagates to a replica faster than A, a reader can see B without A, which looks broken to a user. ## You cannot simply delete the lag There is no free way to eliminate lag entirely without giving up the throughput/latency benefit of async replication. The alternative, synchronous replication, has the primary wait for the replica to apply the write before acknowledging - this trades write latency and availability, since a slow or down replica now blocks writes, for zero lag. Most production systems instead solve staleness at the application or routing layer rather than eliminating it at the source. ## Concrete mitigation techniques Concrete mitigation techniques - (1) read-your-writes routing, (2) sticky sessions, (3) bounded-staleness reads, (4) client-side consistency tokens, (5) return the data the client just sent: 1. **Read-your-writes routing** - after a write, route that user's subsequent reads to the primary for some window, or track the write's log sequence number/commit timestamp and have the read wait until a replica reports it has applied at least that position (Postgres exposes this via `pg_last_wal_replay_lsn()`; MySQL via GTID position checks). 2. **Sticky sessions / monotonic reads** - pin a user's session to a single replica (e.g. via consistent hashing on session ID) so they only ever move forward in time, never backward. 3. **Bounded-staleness reads** - have the load balancer or client library check a replica's lag metric and exclude any replica lagging beyond an acceptable threshold, e.g. one second, from the read pool. 4. **Client-side consistency tokens** - some managed databases (e.g. DynamoDB's `ConsistentRead` flag, or Spanner's read timestamps) let the client explicitly request a strongly consistent read when freshness matters, and fall back to cheaper eventually-consistent reads elsewhere. 5. **Return the data the client just sent** instead of re-reading it from the database - often the simplest fix, since the client already knows what it just wrote and doesn't need a round-trip to confirm it. ## How it shows up in production In production, replication lag shows up as a support ticket, not an error message: a user updates their profile picture, refreshes, and still sees the old one; a user places an order and the order-confirmation page, read from a replica, briefly says 'order not found.' Teams that don't design for it end up bolting on ad hoc fixes - e.g. 'always read orders from the primary for the first five seconds after creation' - which is exactly the read-your-writes pattern above, just discovered the hard way. This exact bug pattern (a user's own post briefly disappearing after posting because the detail page read from a lagging replica) is common enough in social and e-commerce apps that routing post-write reads to the primary, or to a confirmed-caught-up replica, is now considered standard practice rather than a special case.