A service pages the on-call engineer whenever a host's CPU stays above 85% for five minutes, and users have never noticed anything during those pages. In alerting design, what is the difference between a symptom-based and a cause-based alert, and which of the two belongs on the pager?

## The two kinds of alert Every monitoring condition you write sits somewhere on a line between two poles. A **symptom-based** alert fires on evidence that the service is failing at the point where a user or a calling system touches it. Typical forms: the fraction of requests returning server errors over the last few minutes, the 99th-percentile response time of a public endpoint, the age of the oldest unprocessed item in a work queue, the fraction of a service level objective's budget consumed so far. These conditions say *something is wrong for someone*. A **cause-based** alert fires on internal machine state that is one or more steps removed from that experience: CPU utilisation, memory used, disk free, connection-pool occupancy, garbage-collection pause time, a process restart counter, replica count. These conditions say *something is unusual inside the box*. The distinction is not about which metric is more useful — cause metrics are indispensable for diagnosis. It is about which one earns the right to wake somebody up. ## Why paging on symptoms wins Three arguments, in the order interviewers usually want to hear them. **Coverage.** The set of causes of an outage is open-ended and you cannot enumerate it. A bad deploy, a poisoned cache, a dependency's timeout, a certificate, a leap-second bug, a noisy neighbour on shared hardware — each has its own cause signal, and you will not have thought of all of them. But every one of them, if it matters, shows up in the same small set of user-visible measurements. One alert on the error ratio of the checkout endpoint covers causes nobody has invented yet. **Actionability.** A page is a claim that a human must act now. CPU at 92% with normal latency and no errors is not a claim of harm; it is a claim about headroom, and headroom is a planning problem with a calendar, not a pager. When a page arrives that requires no action, the responder learns something dangerous: that pages can be ignored. Precision loss is cumulative and hard to undo. **Correctness under redundancy.** Modern systems are built so that individual components can be sick without users noticing. A single node pegged at 100% CPU behind a load balancer with nine healthy peers is exactly the design working. A cause-based pager punishes you for building redundancy, because it fires on every degraded component whether or not the redundancy absorbed it. ## Where the golden signals fit The common taxonomy for symptom signals is latency, traffic, errors and saturation. Three of the four — latency, errors and, when it collapses to zero, traffic — describe what a request experienced, and are natural paging material. Saturation is the odd one out: it measures how full a constrained resource is, which is a cause. It is the leading indicator you want on a dashboard and in capacity reviews, not the thing you page on by default. ``` page : error ratio, tail latency, request rate collapsed to zero, objective budget burning down ticket : CPU, memory, disk trend, pool occupancy, restart counts, single-replica failures absorbed by redundancy log only: everything you want during a postmortem and nothing else ``` The three-way split — page, ticket, log — is the practical form of the rule. Nothing gets deleted; things get demoted. ## Working the CPU example Apply the test to the alert in the question. Is any user harmed? No — latency and error rate are normal, so the answer is no by measurement, not by assumption. Is there an action only a human can take right now? No — the action is to decide whether to add capacity, and that decision has days of runway. Therefore: demote. Keep the CPU series, put it on the capacity dashboard next to the request rate that drives it, and file a ticket when the trend crosses a planning threshold. Then ask the question the CPU alert was standing in for. Probably: *will this host run out of headroom and start dropping requests?* That is answerable directly by measuring the dropped requests and the latency, which is a symptom, and which will fire whether the cause turns out to be CPU, a slow dependency or a lock. ## The honest exception The rule is not absolute. A cause deserves a page when the impact is certain and the lead time is short relative to the time it takes to fix — a disk that will be full within the hour, a certificate expiring tonight. That is a narrow, justified list, not a licence to keep the old thresholds. ## What the interviewer is listening for Candidates who have only read about this recite "alert on symptoms". Candidates who have carried a pager say what they did with the cause signals afterwards — that they were kept, that they were routed to tickets and dashboards, and that during an incident the symptom page tells you *that* you are broken while the cause metrics tell you *where*.