Why does MongoDB rewrite operations into idempotent form before writing them to the oplog?

## What the oplog is The oplog ("operations log") is a special **capped collection** named `oplog.rs`, stored in the `local` database of every data-bearing replica set member. Capped means it is a fixed-size ring: once full, the oldest entries are overwritten to make room for new ones. `local` is never itself replicated, so each member owns its own oplog. The primary appends an entry for every change it applies. Each entry carries, among other fields, a timestamp `ts` that uniquely orders it, an operation type `op` (`i` insert, `u` update, `d` delete, `c` command, `n` no-op), the namespace `ns`, and the operation payload `o` (plus `o2` carrying the `_id` for updates). ## Why replay is not exactly-once Secondaries replicate by continuously tailing their sync source's oplog and applying the entries. Nothing in that flow gives exactly-once semantics: - A secondary restarts. It resumes applying from the last point it knows is durable on disk, which may be slightly behind what it actually applied, so the tail gets replayed. - A secondary switches sync source after a failover or a network problem and re-reads overlapping entries. - The applier crashes mid-batch and re-runs the batch from its start. Because replay is normal rather than exceptional, replication correctness cannot depend on each entry being applied exactly once. It depends on each entry being **idempotent**: applying it once and applying it five times must leave the document in the same state. ## How MongoDB achieves it The key move is that the oplog does not record the *statement the client sent*; it records the *effect on each document*, in absolute terms. **Relative operators become absolute values.** An application that runs `{ $inc: { qty: 1 } }` against a document where `qty` was 11 does not produce an oplog entry containing `$inc`. It produces an entry that sets `qty` to 12. Replay that entry a hundred times and `qty` is still 12. Had the literal `$inc` been logged, each replay would have incremented again and the secondary would silently diverge from the primary. **Multi-document writes are fanned out.** A single `updateMany` or `deleteMany` that touches 500 documents does not become one oplog entry. It becomes 500 entries, each targeting one document by `_id` in the `o2` field. This matters for two reasons: a re-run of a partially applied batch converges, and an entry never depends on re-evaluating a query predicate whose matching set might have changed since. **Non-deterministic values are resolved at logging time.** Anything the primary computed — a generated `_id`, a server-side date, the result of an upsert deciding to insert rather than update — is baked into the entry as a concrete value, so the secondary does not recompute and get a different answer. **Recent versions log a compact delta.** Newer MongoDB releases record updates as a versioned delta (a `$v: 2` entry) rather than a full `$set` document, to save space on wide documents. The encoding changed; the property did not — the delta still expresses the resulting field values, not a relative adjustment. ## What idempotency does not give you Idempotency is per-entry. It does not mean order is irrelevant: entries affecting the same document must be applied in oplog order, and MongoDB's applier — which uses multiple writer threads for throughput — guarantees exactly that per document while parallelising across documents. Nor does idempotency make the oplog a general-purpose audit log: it is a fixed-size ring that overwrites its oldest entries, so it is not a durable history of everything that ever happened. ## Why interviewers ask this It is the cleanest single question that proves a candidate understands *how* replication works rather than that it exists. The tell of a strong answer is the `$inc` example: recognising that logging the user's operator verbatim would be a correctness bug, and that logging the outcome instead is what makes crash recovery a simple replay. It also sets up everything downstream — change streams, initial sync, and rollback all rest on the same log.