Latency on a production Atlas cluster spiked after a release — which Atlas tools do you open, and in what order?

## The four tools and what each is for Atlas ships a layered set of observability tools, and part of the interview is knowing which answers which question. **Real-Time Performance Panel (RTPP)** — a live view of the cluster: current operations, network throughput, hottest collections, and the ability to terminate a specific operation. This is the triage instrument. If a single unbounded aggregation is pinning CPU right now, RTPP is where you see it and where you kill it. It shows the present, not the past. **Metrics** — time-series charts per node. Opcounters, query targeting, connections, disk IOPS and latency, cache usage, replication lag and oplog window. This is where you answer *when* and *what kind*: did the regression start exactly at the deploy, is it CPU or disk, did connections climb, is a secondary falling behind. Correlating the change point with a deployment timestamp is often the whole diagnosis. **Query Profiler** — slow operations captured over roughly the last day, plotted by duration, each with operation type, namespace, duration, `keysExamined`, `docsExamined`, response length and the query shape. This converts "the cluster is slow" into "this predicate on this collection is scanning hundreds of thousands of documents". **Performance Advisor** — analysis on top of those slow operations, producing ranked index recommendations with an estimated impact, plus suggestions to drop unused or redundant indexes and warnings about schema anti-patterns such as unbounded arrays or excessive collection counts. It is a recommendation engine, not a measurement. ## The order that works 1. **Is it happening right now?** Open RTPP. If yes, identify the hot namespace and, if something is clearly pathological, kill it to stabilise. Stabilise before you investigate. 2. **When did it start and what moved?** Metrics. Line the change point up against the release. Look at query targeting first for a post-deploy latency regression — a jump in scanned-to-returned is the single most diagnostic signal available, because it says the workload is now reading far more than it returns. 3. **Which operations?** Query Profiler over the window containing the change point. Sort by duration, look for shapes that were not there yesterday, and read `docsExamined` against `nreturned`. 4. **What is the fix?** Performance Advisor for the ranked index suggestion, then verify it against the actual query with `explain()` before creating anything. Index creation on a large production collection is itself an operational event, so it deserves a check rather than a click. ## Blind spots to name The most valuable thing to say at senior level is where these tools do not look. - **The profiler only sees slow operations.** Operations that never exceed the slow-operation threshold are invisible to both the Query Profiler and Performance Advisor. A query taking 40 ms but executed thousands of times per second can dominate CPU and never appear. Atlas adjusts that threshold automatically by default, and you can pin it to a fixed value when you need to see faster operations — at the cost of more logging. When the profiler is empty but the cluster is busy, this is usually why, and the answer is to look at opcounters and RTPP's hottest collections instead. - **Performance Advisor optimises the queries it saw.** It will happily recommend an index for a query that should not be running at all. It has no view of your application; a suggestion that adds a large index to fix a report nobody reads is worse than deleting the report. - **Indexes are not free.** Every added index costs write throughput, cache residency and disk. Performance Advisor's drop-index suggestions exist because teams accumulate them; treat additions with the same scrutiny. - **Not every latency problem is a query.** Disk latency saturation, connection storms from a misconfigured pool, a secondary lagging and dragging majority writes, or a tier that is simply undersized all show up in Metrics rather than the profiler. ## Closing the loop with alerts Diagnosis after the fact is the weaker half. Atlas alerting exists so the same signals page you. The conditions worth configuring on a production cluster are the ones that precede user-visible failure: **Query Targeting: Scanned Objects / Returned** above a threshold (the missing-index alarm), **Replication Oplog Window** dropping below a safe number of hours (initial sync and recovery risk), disk space percentage used, connections as a percentage of the configured limit, and system CPU. Alerts are configured per project and can be routed to email, SMS, Slack, PagerDuty, Opsgenie, Datadog, Microsoft Teams, or a generic webhook, so they land in the same on-call flow as everything else. Configuring these once, in code through the Admin API or Terraform, is what turns the four tools above from an incident-time scramble into a routine. The answer an interviewer wants is this shape: stabilise with RTPP, locate in time with Metrics, name the operation with the Query Profiler, get a candidate from Performance Advisor, verify with `explain()`, then leave behind an alert so the next occurrence announces itself.