Which HTTP calls does a CouchDB replication make to move changes from source to target?
answer
- One direction per replication, always
- Target is asked what it lacks
- _revs_diff runs before any document transfer
- new_edits false preserves revision ids
- _local documents hold the checkpoint
basics
~20 sReplication reads the source's _changes feed, asks the target's _revs_diff which of those revisions it lacks, fetches the missing ones with open_revs, and posts them to the target's _bulk_docs with new_edits set to false. Progress is checkpointed in _local documents.
solid answer
~40 sA replication is one-directional and runs entirely over the public HTTP API. It first checks both peers, optionally creating the target, then computes a **replication id** from the source and target and the replication's own parameters, and reads `_local/<replication-id>` from **both** ends to find a shared checkpoint. From that sequence it reads `_changes` with `style=all_docs`, batches the ids and revisions, and POSTs them to the target's `_revs_diff`, which replies with only the revisions the target is missing. Those it fetches with `GET /db/id?open_revs=[...]&revs=true` (multipart when there are attachments) and writes with `POST /db/_bulk_docs` using `new_edits: false`, so the source's revision ids and ancestry are preserved verbatim rather than new revisions being minted. Periodically it writes an updated checkpoint to `_local` on both peers. Bidirectional sync is simply two of these running in opposite directions.
code
json · 5 lines// POST /target/_revs_diff
{"order:912": ["4-cf9b"], "order:44": ["9-11ae", "9-77dd"]}
// response: only what the target is missing
{"order:912": {"missing": ["4-cf9b"], "possible_ancestors": ["3-a71f"]}}go deeper
Know that replication is plain HTTP, that it runs in one direction, and that two-way sync means two replications. Be able to say it is incremental and resumable rather than a full copy each time.
Walk the sequence: _changes, _revs_diff, open_revs, _bulk_docs with new_edits false, checkpoint. Explain what _revs_diff saves and why preserving revision ids keeps replicas identical.
Diagnose real behaviour: a replication that restarts from zero after a filter edit, why checkpoints live in _local documents on both peers, and how to read _scheduler/jobs when a durable replication is crashing.
Own the topology — hub and spoke versus mesh, how many continuous replications a cluster can carry, where filters or selectors belong, and how replication load interacts with compaction and indexing windows.
## Why it is only HTTP CouchDB replication is a documented protocol built from ordinary requests against the same API applications use. There is no binary wire format, no cluster membership, no special port. That is the reason PouchDB in a browser, or any other implementation, can be a full replication peer with a real CouchDB server: it just has to make the same calls in the same order. A single replication is **one-directional**: it copies changes from a source to a target. Two-way sync is two replications pointed in opposite directions, which is exactly what PouchDB's `sync` starts for you. ## Step 1: verify the peers The replicator issues `GET /source` and `GET /target` to confirm both exist and to read metadata such as `update_seq`. If the replication was configured with `create_target: true` and the target is absent, it issues `PUT /target`. ## Step 2: compute a replication id The replicator derives a stable identifier from the source and target endpoints and from the parameters that affect *what* gets replicated — notably the filter function's source code, the selector, or the `doc_ids` list. The point of hashing the filter's code is correctness: if the filter changes, the set of documents in scope changes, so previously recorded progress no longer means the same thing. The practical consequence bites people: edit a filter function, and the replication id changes, the old checkpoint no longer matches, and the replication restarts from sequence zero. ## Step 3: find the checkpoint The replicator reads `GET /source/_local/<replication-id>` and `GET /target/_local/<replication-id>`. `_local/` documents are ordinary-looking documents with three special properties: they are never replicated, they are not in any index, and they keep no revision history. That makes them the natural place to stash per-peer bookkeeping. A checkpoint document holds a `session_id`, the sequence reached, and a `history` array of earlier sessions. Both ends are compared to find a session id they agree on; the sequence from that shared session is the safe restart point. If they disagree — or either document is missing — the replicator starts from zero. Starting from zero is a performance event, not a correctness one: the change scan and `_revs_diff` still prevent re-transferring documents the target already holds. ## Step 4: read the changes From the resolved sequence, the replicator reads `_changes` with `style=all_docs` so that every leaf revision of every changed document is visible, conflicts included. A continuous replication keeps that feed open; a one-shot replication stops when it reaches the sequence that was current when it started. ## Step 5: ask what is missing The replicator batches the ids and revisions it saw and POSTs them to the target: ```json {"order:912": ["4-cf9b"], "order:44": ["9-11ae", "9-77dd"]} ``` The target answers with only what it lacks, plus revisions it does have that could serve as ancestors: ```json {"order:912": {"missing": ["4-cf9b"], "possible_ancestors": ["3-a71f"]}} ``` This single call is what makes replication cheap to restart and cheap to run against a target that is mostly caught up: the expensive document transfer only happens for revisions the target genuinely does not have. ## Step 6: fetch the missing revisions For each document, the replicator issues `GET /source/{id}?open_revs=[...]&revs=true`, which returns the requested leaf revisions along with their revision ancestry. `revs=true` matters because the target needs the history, not just the leaf, to rebuild the same revision tree. With attachments, the request is made with a multipart `Accept` header and `atts_since` so that attachment bodies the target already has are not resent. ## Step 7: write with new_edits false The revisions are POSTed to `POST /target/_bulk_docs` with `"new_edits": false`. Normally a write mints a new revision id and rejects a stale parent with a 409. With `new_edits: false` the target stores exactly the revision ids and ancestry it was handed. Two consequences follow, and both are load-bearing: - Replicas end up with **identical revision trees**, so every replica independently picks the same winning revision without any coordination. - Replication **never returns 409**. A write that conflicts with what the target already has is stored as an additional leaf of the tree, which is how multi-master conflicts get created rather than lost. ## Step 8: checkpoint again After a batch is durably written, the replicator writes an updated `_local` checkpoint on both peers with the new sequence and session id. Checkpoints are periodic, not per document, so a crash costs you the re-processing of one interval — harmless, because `_revs_diff` will report nothing missing for what already landed. ## Operating it Transient replications are triggered by POSTing to `/_replicate`; durable ones are documents in the `_replicator` database with fields such as `source`, `target`, `continuous`, `create_target`, `filter` and `selector`. The scheduler exposes their state through `/_scheduler/jobs` and `/_scheduler/docs`. Since CouchDB 2.1 the replicator does **not** write replication state back onto the replicator document by default, so those endpoints are where you read it. When a replication mysteriously restarts from the beginning, the first thing to check is whether its filter code — and therefore its replication id — changed.
- What happens if both checkpoint documents are deleted while a large replication is idle?The next run finds no shared session and restarts from sequence zero. It re-scans the whole `_changes` feed and re-runs `_revs_diff` for everything, but the target already holds those revisions so almost nothing is transferred. The cost is I/O and time on both peers, not duplicated or corrupted data, and no revisions are re-created because writes still use the source's revision ids.
- Why does editing a replication's filter function restart it from the beginning?The replication id is derived from the filter's source code along with the endpoints, because a different filter means a different set of in-scope documents. Changing the code changes the id, so the old `_local` checkpoint no longer matches and there is no recorded progress to resume from. Expect a full re-scan whenever you deploy a filter change, and size the maintenance window accordingly.
- Why does replication never return a 409 conflict to the source?Writes go through `_bulk_docs` with `new_edits: false`, which stores the supplied revision ids as-is rather than validating a parent revision. A revision that diverges from what the target holds simply becomes another leaf of the revision tree. That is deliberate: replication must never drop a write, so divergence is preserved as a conflict for the application to resolve.
saying these in an interview costs you the question
- Thinks one replication syncs both directions
- Believes documents are transferred before _revs_diff runs
- Says the target mints fresh revision ids on write
- Looks for checkpoints in the _replicator document instead of _local
- Assumes a lost checkpoint means data loss