Across forty engineering teams, incident severity grades are wildly inconsistent — one team's SEV1 is another's SEV3, and one team has not declared anything above SEV3 in a year. As the lead of the reliability practice, how would you calibrate severity across the organisation?

## The wrong first move The instinctive fix is to rewrite the severity matrix with more precise language. It almost never works, because the teams are not confused about what the words mean. A team that has not declared above SEV3 in a year is not misreading a definition — it is responding rationally to what happens when it declares a SEV2. Severity grading is a behaviour, and behaviours follow consequences. Start with the consequences. ## Diagnose the direction, per team The two failure directions have opposite causes and opposite fixes, and an organisation usually has both at once. **Inflation** — everything is a SEV1. Typically severity has become the only lever that reliably gets attention: the only way to pull in a specialist team, get an exception to a freeze, or make a dependency owner respond. The severity dial is being used as a priority dial. Fix the underlying access problem and inflation subsides on its own. **Deflation** — nothing is ever bad. Almost always the high tiers carry weight that feels punitive: a mandatory review with an audience, executive visibility, a mark that follows the team into performance conversations, or a reliability metric the team is measured on. Sometimes it is simpler and sadder — declaring high wakes people up, and the team does not want to be the team that does that. Either way, the team is paying for honesty, so it stops being honest. The evidence is in the data. Look at grade distribution per team over time, compare it to the same teams' user-facing impact measured independently, and look at how many incidents were declared at all. A team with normal outage evidence but a compressed severity distribution is deflating. ## Calibrate with examples, not prose Humans calibrate against cases far better than against definitions. The single highest-leverage artefact is a **library of worked reference incidents**: real incidents from across the organisation, each with its final grade and two or three sentences of reasoning, including the near-misses where the grade was genuinely arguable. "This looked like a SEV1 but had a documented workaround and stayed at SEV2, here is why" teaches more than a paragraph of tier definition ever will. Pair it with a recurring **re-grading review**: a small cross-team group takes a sample of closed incidents from the last month and independently grades them, then compares. Disagreements are the product — they tell you exactly which boundary is ambiguous and which team is drifting, and the discussion itself is the calibration mechanism. Publish the disagreements, not just the conclusions. ## Change the structure, not just the guidance **Decouple burden from tier.** Make declaring a high severity organisationally cheap and only *analytically* expensive. A serious incident should earn a thorough review; it should not earn a hostile audience. If a tier triggers something the team experiences as punishment, you have built a machine that manufactures under-declaration, and no amount of encouragement will out-argue it. **Separate impact severity from response urgency.** These are genuinely different axes and collapsing them into one number forces bad choices. A large data-correctness problem that has already stopped may deserve the top severity for review and notification purposes while needing no overnight response at all; a moderate but rapidly growing outage is the reverse. Two fields, each with its own meaning, removes the incentive to misgrade impact in order to control who gets woken. **Normalise breadth by service criticality.** An absolute user-count threshold cannot be shared between a payments service and an internal reporting tool. Establish criticality tiers for services first, then express severity thresholds relative to that tier. This is also what makes cross-team comparison meaningful, because you are comparing like with like. **Keep the taxonomy small and centrally owned.** Per-team matrices guarantee divergence. One matrix, owned by the reliability practice, with a documented path to propose changes. ## What to accept Some variance is legitimate and chasing it is waste. Domains genuinely differ: a research-facing service and a regulated payments flow will apply the same principles to different consequences. Perfect inter-rater agreement is not the goal — the goal is that a given grade means roughly the same *urgency and obligation* everywhere, so that a person from another team reading it knows what to do. And be honest about the timeline. This is a trust problem, and the deflating team will keep deflating until it has watched a couple of high-severity declarations happen to other teams without consequence. Removing the punishment is necessary but not sufficient; the evidence has to accumulate publicly before the behaviour moves.