In MongoDB's WiredTiger engine, what do a checkpoint and the journal each do, and how do they work together for durability?
answer
- one is a recovery point
- the other covers the gap
- 60 seconds against 100 milliseconds
- old journal files can go after a checkpoint
- a replica set journals less
basics
~20 sA checkpoint durably writes a consistent snapshot to the data files, by default about every 60 seconds; the journal is a write-ahead log synced about every 100 ms. Recovery starts at the last checkpoint and replays the journal.
solid answer
~50 sA checkpoint is WiredTiger writing a snapshot's data consistently across all data files, which makes those files a valid recovery point. `storage.syncPeriodSecs` defaults to 60 seconds, and the server's option comment measures that from the end of the previous checkpoint. The old checkpoint stays valid until metadata is atomically updated to reference the new one. The journal is a write-ahead log under `dbPath/journal`: records are buffered in memory and synced every `storage.journal.commitIntervalMs`, 100 ms by default, or sooner on a `j: true` write. After a crash, MongoDB opens the last checkpoint and applies the journal from that point. Checkpoints bound replay time and let old journal files be removed; the journal narrows the loss window from a checkpoint interval to milliseconds. On a replica set member the journal covers the oplog rather than replicated collections, and replaying the oplog rebuilds the rest.
go deeper
Recall the two roles: a checkpoint is a consistent point on disk, and the journal records what happened since. Know the defaults of 60 seconds and 100 milliseconds.
Walk through a restart step by step, and explain why a crash in the middle of a checkpoint is safe: the previous checkpoint stays valid until metadata switches atomically.
Show you know the journal's scope differs by deployment: a standalone journals everything, while a replica set member journals the oplog and relies on oplog replay for the rest.
Frame both intervals as things to leave alone. They trade recovery time, disk and small loss windows, and the durability a team really cares about is expressed through write concern.
## Two mechanisms, one durability story WiredTiger keeps recently changed data in memory, because writing every change to its final place in the data files would be far too slow. That leaves two questions: what is on disk that is known to be consistent, and how are the changes since then protected? A **checkpoint** answers the first and the **journal** answers the second. ## What a checkpoint is A checkpoint is a durable, consistent view of the data. WiredTiger takes a snapshot and writes all the data in it to disk in a consistent way across all of the data files. Some details matter when you explain it: - **It is not a full copy.** Dirty pages are written; clean pages are skipped. - **It has a default interval.** `storage.syncPeriodSecs` defaults to 60 seconds. The comment beside the option in the 9.0 source calls it the delay before triggering the next checkpoint after the previous one completes, so the gap between starts is 60 seconds plus however long the checkpoint took. - **The switch is atomic.** While a new checkpoint is being written, the previous one remains valid. The new one takes over when WiredTiger's metadata is atomically updated to reference it, and only then are pages from older checkpoints freed. - **A crash mid-checkpoint is safe.** MongoDB recovers from the last complete checkpoint. Checkpoints alone would give a consistent database after a crash, but one that had lost everything since the last checkpoint: up to a minute or more of writes. ## What the journal holds The journal is WiredTiger's **write-ahead log**, stored in `WiredTigerLog.<sequence>` files in a `journal` directory under `storage.dbPath`. It is not the MongoDB log file that holds diagnostic messages. - A journal record describes one client-initiated write, together with the internal changes it caused, such as index updates. - Records go into an in-memory buffer first. - WiredTiger syncs that buffer to disk every `storage.journal.commitIntervalMs` (100 ms by default), when a write includes or implies `j: true`, and when it creates a new journal file, which happens about every 100 MB. - On a secondary it also syncs after each batch of oplog entries it applies. - Records are compressed with `snappy` by default. ## How they combine at restart On restart after an unclean shutdown, recovery runs before the server becomes available: 1. MongoDB looks in the data files for the identifier of the last checkpoint. 2. It finds the matching record in the journal files. 3. It applies the operations recorded in the journal since that point. The relationship also runs the other way. Journal files older than the most recently completed checkpoint are removed, so checkpoints limit both how much journal must be replayed and how much disk it occupies. WiredTiger's own documentation gives exactly these two reasons for checkpointing when a log is enabled. | | Checkpoint | Journal | |---|---|---| | Format | Consistent data files | Sequential log records | | Default frequency | About 60 seconds | About 100 ms | | Setting | `storage.syncPeriodSecs` | `storage.journal.commitIntervalMs` | | Job in recovery | The starting point | The replay since that point | ## Standalone against replica set member The docs describe one journal record per client write, and that picture fits a standalone. On a replica set member the source is narrower. `WiredTigerUtil::useTableLogging` turns table logging off for replicated collections and keeps it on for the `local` database, including the oplog (`local.replset.minvalid` is the exception). A replica set member therefore relies on this sequence: - The journal protects the oplog entries. - A stable checkpoint holds the collections up to a timestamp. - Replaying the oplog past that timestamp restores the rest, which belongs with replication rather than with storage. An answer that says the journal holds every collection write on a replica set is repeating a simplification. ## Telling them apart from similar things - The **oplog** is a replicated collection of operations for other members to apply. The journal is a local storage-engine log. - `fsync` asks for pending writes to be flushed to disk, and is not the periodic checkpoint schedule. - `j: true` asks the server to acknowledge a write only after it reaches the on-disk journal. What it promises an application is a write concern question; for storage, it simply triggers an immediate sync.
- With WiredTiger on MongoDB, why keep checkpoints at all if the journal records every change?Without them, journal files could never be removed and recovery would have to replay everything. Log files are only removed once a checkpoint completes, and each checkpoint limits the records a restart must process. MongoDB's docs warn that setting `storage.syncPeriodSecs` to 0 lets the journal eventually consume all available disk space.
- Why does the documented procedure for changing WiredTiger's journal compressor in MongoDB start from a clean shutdown?After an unclean shutdown, recovery has to read the existing journal files, and the manual says you must use the old compressor settings for that. So the procedure stops `mongod` cleanly, changes `journalCompressor` and restarts. If an unclean shutdown happens midway, start with the old setting, let it recover, then retry.
- How do you see journal activity on a running mongod?`db.serverStatus().wiredTiger.log` returns statistics on the journal. The files themselves are `WiredTigerLog.<sequence>` under `dbPath/journal`, along with pre-allocated log files WiredTiger keeps ready for use.
A writer saves a clean copy of a long document once a minute and, between saves, notes each edit in a bound logbook several times a second. After a blackout she opens the last clean copy and reapplies the logbook. Only the edits she had not yet written down are gone.
- An acknowledged write is on disk only once the next checkpoint has run.
- The journal is the same thing as MongoDB's diagnostic log file.
- storage.syncPeriodSecs controls how often the journal is synced.
- A crash during a checkpoint corrupts the data files until you run repair.
- The journal and the oplog are the same log under two names.
- A checkpoint rewrites a complete copy of the database every minute.