A drift alarm flipping on and off refits the hotel pricing model six times in one day, so how do you stop that?

## Why a bare threshold chatters A drift statistic computed over a rolling window is noisy by construction: the window rolls, the sample changes, and a value hovering near its threshold crosses it repeatedly. If the trigger is literally "on the alarm's rising edge, start a refresh", every crossing is a run. Nothing about the model changed between 10:00 and 14:00 — only which side of a line a noisy number landed on. This is the classic shape of an undamped control loop, and it is fixed the same way it is fixed everywhere else: put distance between the condition that fires and the condition that resets, and put a floor under how often the actuator can move. ## Four dampers, and what each one actually prevents | damper | rule | what it prevents | |---|---|---| | hysteresis | fire above X, clear only below Y where Y is meaningfully below X | crossings from noise around a single line | | persistence | the alarm must hold for k consecutive windows | a single spiky window firing a run | | minimum interval | no refit within N hours of the last one | a legitimate but sustained alarm refitting continuously | | run cap | at most M runs per day, and page when the cap is hit | unbounded spend and pipeline contention, silently | They are complementary, not alternatives. Hysteresis and persistence damp the **signal into the trigger**; the minimum interval and the cap bound the **actuator** whatever the signal does. A system with only the first pair still refits every hour when drift is genuinely sustained; a system with only the second pair still burns its whole daily budget on noise. One detail matters in the implementation: when an alarm fires inside the cooldown, **coalesce** it — record that a refresh is pending and run once when the interval expires. Queueing each edge instead means the cooldown only delays the storm. ## What the flapping costs It is tempting to treat this as a spend problem. It is more than that: - **compute and contention** — the retraining jobs crowd out the scheduled run and whatever else shares the batch capacity; - **model-version churn** — six unreviewed model versions reached production pricing in a day, each fit on a slightly different window; - **rate volatility** — guests and revenue managers see the portfolio's rates move for reasons no one can explain; - **lost reasoning** — asking "what priced this room night?" becomes an archaeology exercise across near-identical versions; - **the gate under load** — an automatic promotion gate tuned for a daily candidate is being asked to judge six, and its own evaluation noise now matters. ## When the alarm is the symptom, not the trigger Before damping anything, ask why the statistic is unstable. Three causes are far more common than a genuinely oscillating market: 1. **A flaky upstream feed** that intermittently drops or freezes a field — the input distribution really does move, but the fix is in the pipeline, not the model. 2. **A window too short for the traffic volume**, so the statistic is dominated by sampling noise, particularly on small properties or thin stay-date segments. 3. **A reference that has itself moved** — a rolling reference window quietly adopts the drift it is meant to catch, so the comparison swings as the reference catches up. In the first case, refitting is actively harmful: the run fits a new model version on the corrupted window and the defect moves from the feed into the artifact, where it survives the feed being fixed. The trigger's first stop should therefore be a data-quality check on the training window, and the alarm payload should say *which* features moved, because one feature moving alone is a plumbing tell and the whole vector moving together is a market tell. ## A trigger policy that holds Put together, the policy that survives contact with production reads: 1. The drift alarm must exceed its fire threshold for **k consecutive windows** and clear only below a lower threshold. 2. On firing, run a **data-quality check** on the training window; on failure, page a human instead of starting a refresh. 3. If a refresh is already within the **minimum interval**, mark one pending and coalesce. 4. Enforce a **daily cap**, and treat hitting it as an incident rather than as normal operation. 5. Keep the **clock cadence** underneath as the staleness ceiling, unchanged by any of the above. The last line of defence is not part of this leaf's job, but it is worth naming in the answer: even with a well-damped trigger, what stops a bad refresh from pricing rooms is the standing gate every candidate clears before promotion.