skip to content

Validate, Profiler & Server Stats

The validate command checks a collection's structures, the profiler and slow-query log find bad queries, and serverStatus, currentOp and mongostat show load. A staple of operations rounds.

part ofMongoDBread it →
on this pageshow

explore

questions

6

In MongoDB, what do profiling levels 0, 1 and 2 record, and how do `slowms`, `sampleRate` and the slow-query log relate to them?

level: middleimportance: must knowfreq 52%

answer

  1. off, slow, everything
  2. two destinations, one threshold
  3. level per database, threshold per process
  4. the log keeps working at level 0
  5. a small capped collection

basics

~20 s

Level 0 records nothing in system.profile, level 1 records operations at or above slowms or matching a filter, and level 2 records everything. Separately, slowms and sampleRate also drive the slow-query log, which keeps working at level 0.

solid answer

~40 s

The profiler writes to a capped `system.profile` collection per database: level 0, the default, writes nothing; level 1 writes operations at or above `slowms`, default 100 ms, or those matching a `filter`; level 2 writes everything. The trap is that `slowms` and `sampleRate` also control the diagnostic log, so at level 0 slow operations still appear there as `Slow query` lines. Scope differs as well: `db.setProfilingLevel` sets the level and filter for the current database, but `slowms` and `sampleRate` apply to the whole `mongod`. The level resets on restart unless it is in the config file. Since 8.0, slowness is judged on `workingMillis`, so time spent waiting on locks does not count. A `mongos` has no profiler, so these settings affect only its log.

code

javascript · 10 lines
javascript
// Level 1 for this database; slowms and sampleRate for the whole mongod
db.setProfilingLevel(1, { slowms: 200, sampleRate: 0.5 })

db.getProfilingStatus()

// Most recent slow operations in this database, on this member
db.system.profile.find({ millis: { $gte: 200 } }).sort({ ts: -1 }).limit(10)

// Back to off. The slow-query log carries on.
db.setProfilingLevel(0)

go deeper

for a junior

Recall the three levels, that level 0 is the default, that slowms defaults to 100 milliseconds, and that entries go to the system.profile collection in each database.

for a middle

Explain that one threshold feeds two destinations, why the log keeps reporting at level 0, which settings are per database and which per process, and what a filter overrides.

for a senior

Show you would reach for the slow-query log first, know the 1 MB capped collection loses history under load, and account for workingMillis hiding lock waits since 8.0.

for a principal

Treat it as a budget: every profiled operation adds a write and may record query values, so decide how much detail is worth paying for and for how long.

## The profiler and its levels The **database profiler** records details of operations on a `mongod`: CRUD, plus configuration and administration commands. It writes them to `system.profile`, a capped collection in each profiled database, created when you enable profiling there. | Level | `operationProfiling.mode` | What goes to `system.profile` | |---|---|---| | `0` | `off` | Nothing. This is the default | | `1` | `slowOp` | Operations at or above `slowms`, or those matching `filter` if one is set | | `2` | `all` | Every operation; `slowms` and `filter` are ignored by the profiler | The command `db.setProfilingLevel(level, options)` changes this at runtime and replies with the **previous** settings: `was` is the old level, not the new one. Check the current state with `db.getProfilingStatus()`. A level set this way does not persist; after a restart the server goes back to `0` or to what the config file or `--profile` says. ## The second destination: the slow-query log This is the part candidates often miss. Two destinations share one threshold: - **`system.profile`**, which only receives data when the level is 1 or 2. - **The diagnostic log**, which at the default `logLevel` of 0 writes operations at or above `slowms` as `Slow query` messages (id 51803), at the rate `sampleRate` allows. This happens **at every profiler level, including 0**. So you can find slow operations with the profiler off. At higher `logLevel` values, all operations reach the log whatever their latency. ## Scope and defaults | Setting | Default | Scope when set with `db.setProfilingLevel` | |---|---|---| | level | `0` | The current database | | `filter` | none | The current database | | `slowms` | `100` ms | Every database in the process | | `sampleRate` | `1.0` | Every database in the process | Set in the config file or on the command line, level and filter apply to the whole process. A runtime change reaches only the process you are connected to, so a replica set needs it on each member, and a change made by hand is easy to lose track of when members restart or fail over. `sampleRate` samples **slow** operations for both destinations. It accepts values from 0 to 1 inclusive; at `0.5`, about half of the slow operations are recorded. `filter` replaces the threshold logic. A predicate on any profiler output field decides what is profiled and logged, and while it is set, `slowms` and `sampleRate` are not used for either destination. A filter on a `mongod` thus changes what the slow-query log shows too. ## What counts as slow Since MongoDB 8.0, slow-operation logging is based on **`workingMillis`**, the time MongoDB actually spends working on the operation. Waiting for locks or flow control does not push an operation over the threshold. An operation stuck behind a lock can therefore be very slow for the client and still not be reported as slow. A filter can use `workingMillis` and `durationMillis` with separate thresholds if you need to catch those. In 8.3 and later, `slowinprogms` (default 5000) adds a smaller log entry, at most once per query, while a query is still running, which helps with operations that never finish. ## What it costs - **Writes.** Each profiled operation adds a write and needs a lock, which is why the manual warns the profiler can degrade performance, especially at level 2 or at level 1 with a low threshold. - **Space and retention.** `system.profile` defaults to 1 MB, enough for several thousand entries by the manual's estimate. On a busy database, the entry you want may already be gone, so resizing means turning profiling off, dropping the collection and creating a larger capped one. - **Member-local data.** Profile writes are not replicated, so each member has its own `system.profile`. - **Sensitive data.** Entries can hold query values, and the manual warns the profiler can expose unencrypted query data. ## A `mongos` and transactions A `mongos` has no profiler: the level must stay `0`, and `slowms`, `sampleRate` and `filter` only affect its log. To profile a sharded cluster, enable it on each `mongod`. And `system.profile` cannot be read inside a transaction. Once you have found a slow operation in either place, the entry gives you its namespace, duration, `keysExamined`, `docsExamined` and `planSummary`. Reading its plan with `explain()` is the next, separate step.

  • Why might an operation a client saw take 30 seconds not show up as slow in the log of a MongoDB 8.0 server?
    Since 8.0, the slow threshold is checked against `workingMillis`, which leaves out time spent waiting for locks and flow control. An operation that mostly queued behind a lock can sit under `slowms`. To catch those, set a `filter` that also tests `durationMillis`, and look at live operations for waits as they happen.
  • If you set a profiling `filter`, what happens to `slowms`?
    It stops being used for both the profiler and the slow-query log, and so does `sampleRate`. The server even adds a note saying so to the `db.setProfilingLevel` reply. Only operations matching the filter are recorded, which is easy to forget when the log suddenly goes quiet. Pass `filter: "unset"` to the `profile` command to remove it.
  • How large is `system.profile` by default, and how do you change it?
    It is a capped collection of 1 MB by default. To resize it on a primary, set the level to 0, drop `system.profile`, create it again with `capped: true` and a larger `size`, and turn profiling back on. The size is per database and per member, so a secondary has to be done separately, per the manual, by running it as a standalone.

  • At profiling level 0, MongoDB reports slow operations nowhere at all
  • slowms applies only to the database you ran db.setProfilingLevel on
  • A profiling level set with db.setProfilingLevel survives a restart
  • The was field in the reply shows the level you just set
  • You can enable the profiler on a mongos to profile a sharded cluster
  • system.profile grows without limit, so level 2 mainly costs disk space

Assumes MongoDB 9.0; judging slowness on workingMillis dates from 8.0, while slowinprogms arrived in 8.3, so an 8.0 deployment has the first and not the second.

open as a page

What do MongoDB's `mongostat` and `mongotop` each show, and where do their numbers come from?

level: juniorimportance: should knowfreq 27%

basics

~20 s

mongostat prints server-wide rates such as inserts, queries and connections per interval, built from serverStatus. mongotop prints milliseconds of read and write time per collection, built from the top command. Together they show how busy and where.

open as a page

What does MongoDB's `validate` command check on a collection, what does it lock, and what do `full` and `background` change?

level: middleimportance: should knowfreq 24%

basics

~10 s

MongoDB's validate scans one collection's documents and indexes for damage on one member. By default it holds an exclusive collection lock; full adds storage-engine verification, while background reads the last stable checkpoint without blocking.

open as a page

MongoDB API calls turn slow right now: how do you use `serverStatus`, `$currentOp` and `killOp` to find and stop the cause, and what can go wrong?

level: seniorimportance: should knowfreq 44%

basics

~20 s

Sample serverStatus twice to see the load, list live work with $currentOp and allUsers, then kill a confirmed client operation with killOp. The kill is only a mark that takes effect at a safe point, so verify it.

open as a page

What should a team running self-managed MongoDB leave on in production: profiler level, slow-query thresholds, and who may kill an operation?

level: principalimportance: should knowfreq 36%

basics

~20 s

A sensible default leaves the profiler off, ships the slow-query log with a threshold tuned to its volume, polls serverStatus, and time-boxes level 1 for investigations. Kills go to a narrow inprog and killop role, not to wide administrative roles.

open as a page

A disk fault is reported on one member of a MongoDB replica set: how do `validate`, `dbCheck` and `repair` each fit into deciding what to do?

level: seniorimportance: nice to knowfreq 15%

basics

~20 s

Run validate on the suspect member to check its own structures, and dbCheck from the primary to compare members through hashed batches. Then resync the damaged member from an intact one; repair is standalone only and removes corrupt documents.

open as a page