Who should be able to pull the dropout early-warning model's kill switch, given that a switch one team alone can reach is not a switch?

## Why one team's switch is not a switch A kill switch is only worth what its **availability** is worth, and availability here is a property of the authority, not of the flag. If the only people permitted to disable a model are the four engineers who built it, then the switch does not exist on a holiday weekend, does not exist while they are on a flight, and does not exist during the hour they spend deciding whether pulling it would look like an admission. The mitigation with the shortest technical path can still have the longest human one. The second failure is the mirror image. A switch anyone can pull with no stated criteria gets pulled for a dashboard that looked odd, and every spurious pull costs real detection quality - students who would have been found early and now are not. Both failure modes are real, and the design has to answer both at once. ## The list, and what each entry is for | Who | Why they are on it | What they need | |---|---|---| | On-call for the serving tier | Reachable at any hour, independent of the model team | Permission and a criterion, not a model expert's sign-off | | Accountable business owner | Can act on harm without a technical argument | A direct route that does not queue behind engineering triage | | The model's owning team | Deepest understanding of the failure | Nothing extra - they are the default, not the gate | The entry that makes the switch real is the first one. It also implies an obligation: if the on-call may pull it, the on-call must be able to *find* it, must know what the system does afterwards, and must not need to discover any of that during the incident. ## Pre-authorised criteria remove the judgment burden The reason correct pulls do not happen is rarely permission on paper. It is that the person holding the switch expects to defend the decision afterwards while the counterfactual - the harm they prevented - is invisible. Fix that by writing the conditions in advance, in terms the on-call can evaluate: 1. A **known-bad input**: an upstream feed confirmed wrong, regardless of what the model's outputs look like. 2. **Published flag volume or composition outside its agreed band** for two consecutive cycles, with no explaining change. 3. **Corroborated reports** of systematically wrong decisions from the people consuming them. 4. Any case where **the cause is unknown and the downstream action is hard to undo**. Under a written criterion, pulling is compliance, not a judgment call - and the review afterwards asks whether the criterion is right, never whether the person was brave enough. ## Make the two directions asymmetric The strongest simplification in this whole design is that **off and on are not the same decision**: - **Off** is single-person, needs no quorum, needs no cause, and is logged. Requiring two approvers to stop harm is a quorum at 02:00, which is the same as not having a switch. - **On** needs the owning team, evidence that the suspected input is verified good, a cycle scored and compared against the fallback, and a decision on whether the affected cycles get re-scored. - The default on a pull is **not** a silent return. An expiring switch should revert to a *review*, not to the model, or a resolved incident quietly re-enables the failure with nobody watching. This asymmetry is the same reasoning as any recoverable-versus-irreversible pairing: turning the model off costs bounded, known detection quality, while turning it back on too early resumes unbounded, unexplained harm. Price the two directions accordingly. ## What the pull leaves behind Record the switch event as its own audit entry - who pulled it, when, which criterion they cited, and what the system served afterwards. This is not the per-prediction decision record; it is a governance record about the control itself, and it has two jobs. It is the input to the review that decides whether the criteria need changing, and it is the evidence that the control the organisation claims to have was actually exercised. A switch nobody can prove was ever pulled is, to an auditor and to a regulator, indistinguishable from one that does not work. ## The cost of widening the list Every name you add raises the rate of unnecessary pulls, and unnecessary pulls are not free: the fallback finds fewer at-risk students, and the model's owners lose confidence in a control they cannot predict. Buy the width back with three cheap things - written criteria, a mandatory notification to the owning team on every pull, and a fast, evidence-based path to restore. The organisation that has agreed all of this in a quiet week can act in ten minutes; the one that has not will spend its incident arguing about who is allowed to decide.