Beyond Merkle-tree anti-entropy, some systems reconcile replicas using log-based mechanisms — for example shipping a replication log, or Kafka-style log compaction that retains only the latest value per key. Compare log-based reconciliation to Merkle-tree anti-entropy: what problem does each solve best, and what does each cost?

## Two families of technique Reconciliation in an eventually consistent system means converging replicas that have drifted apart, and there are two structurally different families of technique for doing it: **log-based**, meaning streaming or replaying an ordered history of changes, and **state-based**, meaning comparing current snapshots, which is what Merkle-tree anti-entropy does. Understanding when each is the right tool, and why serious systems use both, requires understanding what each actually observes. ## Replaying an ordered history Log-based reconciliation works by treating every write as an entry in an ordered, append-only log, a write-ahead log or changelog. A replica that's behind simply needs the log entries it's missing, in order, and replays them to catch up — this is how most primary-replica database replication works, such as PostgreSQL WAL shipping or MySQL binlog replication, and it is also the basis of hinted handoff in Dynamo-style systems, where a coordinator temporarily buffers writes destined for a down replica and replays them once it's back. Log compaction, as implemented in Kafka, is a specific optimization on top of this idea: rather than retaining every historical write forever, a background compaction process periodically rewrites each log segment to keep only the most recent record for each key, discarding older superseded records, with tombstones for deleted keys kept for a configurable grace period before also being purged. This bounds the log's storage growth while still letting a new consumer reading the compacted log from the beginning reconstruct the current state of every key — reconciliation via replaying the minimal sufficient history, not via comparing two already-materialized states. ## The blind spot in the log The core assumption log-based approaches make is that the log itself is the complete, authoritative source of truth and is never silently lost or corrupted in transit. That assumption is exactly where it breaks down in production: - if a replica is offline longer than the log's retention window, - a network issue truncates log delivery, - or a bug in the replication path silently drops a batch, then the destination replica has no log entries telling it what it's missing, because the very thing that would tell it is the thing that got lost. The replica doesn't know it's wrong; it just silently has old data with no error signal. ## What comparing current state adds This is precisely the gap Merkle-tree anti-entropy fills, and why it's a state-based rather than log-based technique: it doesn't care what happened or in what order, it only asks whether two replicas currently hold the same data for a key range right now. Because it works from current state rather than history, it can catch divergence regardless of cause: - a dropped log segment, - corrupted disk write that never went through the normal path, - a bug that mutated data out-of-band, - or a replica that was down longer than any retention window could bridge. This is why Merkle-tree anti-entropy typically runs as a slower, periodic, background safety-net process, such as Cassandra's `nodetool repair`, layered on top of the fast, continuous, log- or hint-based path that handles the common case cheaply. ## Weighing the two against each other The trade-off, put directly: | Approach | Strengths | Limits | |---|---|---| | **Log-based reconciliation** | cheap, low-latency, and naturally preserves write ordering, important for monotonic-writes-style guarantees | only as reliable as the log's own durability and retention — it has no way to detect silent data loss on its own | | **State-based, Merkle-tree reconciliation** | comprehensive and self-verifying, eventually finding any drift regardless of cause | comparatively expensive in CPU, IO, and network to hash and exchange tree levels, throws away ordering information since once you're comparing final state, who wrote last has to be resolved by timestamps or vector clocks rather than replay order, and only runs periodically, so it has a detection lag measured in the repair schedule's interval rather than real time | ## Running both together A concrete production pattern: Cassandra's write path uses replication plus hinted handoff, a log-based mechanism where a coordinator holds a buffered hint for a replica that's briefly unreachable and replays it once the replica returns, for fast common-case convergence; separately, operators run scheduled `nodetool repair` using Merkle trees to catch anything hinted handoff missed, for example a replica down longer than the hint's own retention window, which silently drops hints just like Kafka's log compaction would eventually drop older-than-retention records. Without the Merkle-tree safety net, that class of drift would accumulate silently forever.