skip to content

A load forecaster's model-quality alarm fires at 03:00 - what decides whether it pages now or waits until morning?

level: seniorimportance: nice to knowfreq 31%

answer

  1. urgency comes from the decision clock
  2. not from the size of the error
  3. ask what a lever buys tonight
  4. day-ahead has a deadline, not an emergency
  5. committed intervals cannot be repaired

basics

~20 s

The time until the next irreversible commitment on that forecast horizon, not the size of the error. Intraday forecasts are dispatched on continuously, so they page; a day-ahead forecast is only committed at gate closure hours later.

solid answer

~40 s

Tier by the horizon's decision deadline and by whether a mitigation tonight changes anything. An intraday forecaster feeds dispatch and reserve decisions every settlement interval, so degradation at 03:00 is costing money now and a rollback taken now improves every interval still ahead - that is a page. A day-ahead model whose submission closes mid-morning has a deadline, not an emergency: it should raise a ticket that carries an explicit expiry and escalates to a page before gate closure if untouched. Slow decay over weeks is a review item. Note that no mitigation improves forecasts already submitted to the market, so tiers should be set by the volume of commitment still open, not by how bad last night's error looked.

go deeper

for a junior

The takeaway: whether something is worth waking a person for depends on when the next decision is made, not on how large the number on the dashboard looks.

for a middle

Be able to map each forecast horizon to the decision it feeds and say which ones still have an open commitment overnight.

for a senior

Show the middle tier properly: an expiring, escalating, owned item for the day-ahead horizon, and a page only where a lever changes intervals still ahead.

for a principal

The open call is how much of the night you are willing to buy back with burn-in and tiering, given what an hour of degraded intraday forecasting actually costs the operator.

## The tier test on this system The generic principle - page only when a human must act now - becomes concrete on a forecaster because each horizon has a known decision clock. Two questions decide the tier: 1. **When is the next irreversible commitment on this horizon?** Intraday, dispatch and balancing decisions are taken continuously; day-ahead, everything hinges on the submission at gate closure. 2. **Does a mitigation taken now change any commitment still ahead?** A rollback improves the intervals that have not been committed yet, and does nothing for the ones already submitted. If the answer to the first is "within the hour" and to the second is "yes", it is a page. If the deadline is eight hours away and the same action taken at 07:00 buys the same outcome, waking someone bought nothing but fatigue. | horizon | binding decision | degradation found at 02:00 | mitigation available tonight | |---|---|---|---| | next few settlement intervals | dispatch and reserve, continuously | page | rollback or fallback, effective next interval | | day-ahead schedule | submission at gate closure, mid-morning | ticket with a hard expiry | same rollback, equally effective at 07:00 | | week-ahead planning | periodic planning cycle | review item | retrain or refit, out of hours scope | | already-submitted intervals | closed | record only | none - the position is committed | ## Deadlines are not severities The day-ahead row is where teams go wrong in both directions. Paging on it treats a deadline as an emergency and burns the on-call's night for nothing. Filing an ordinary ticket treats it as optional and lets the deadline pass silently, which is the same as having no alarm. What the middle tier needs is structure the ticket normally lacks: - an **explicit expiry** tied to the real deadline, not "soon"; - an **automatic escalation to a page** at a fixed lead time before gate closure if it is still untouched; - a named **owner** for the horizon, so it does not sit in a shared queue; - the same payload the page would have carried, so whoever picks it up at 08:00 is not starting from zero. A ticket that cannot become a page is a silenced alarm with extra steps. ## What makes an intraday page honest A page is honest when the responder has a lever. On this system the levers at 03:00 are narrow and pre-agreed: switch to the previous model version, or fall back to a non-learned baseline. Both take effect on the next interval, and both are reversible. If the alarm fires for a condition where neither lever helps - a corrupted actuals feed that makes the measurement wrong rather than the forecast - then the page should be on the measurement path and worded so the responder does not reach for a rollback that cannot help. This also bounds how many paging rules the forecaster should have: roughly as many as there are horizons with an open commitment and a lever, which is usually one or two, not one per model or per feature. ## Failure modes to name in the interview - **Tiering by error magnitude.** A huge day-ahead error discovered at 02:00 is still not urgent if submission is at 10:00; a modest intraday error is urgent because reserve is being bought against it right now. - **One severity for every horizon** because one model produces them all. The model is shared; the decision clocks are not. - **Expecting a rollback to repair committed intervals.** It cannot. Those become a cost report and, if material, an incident record. - **Night-time scope creep.** Retraining, threshold edits and root-cause analysis are not night work; mitigation is. - **Silent quiet hours.** Suppressing overnight pages wholesale removes the only tier that had a lever.

  • A rollback at 03:00 only improves intervals not yet committed. How should that shape the tier?
    It makes the page's value proportional to the commitment still open ahead of it. For an intraday horizon that volume keeps accruing all night, so acting at 03:00 beats acting at 07:00 by four hours of intervals. For a day-ahead position already submitted, the same action buys nothing until the next submission, which is why that horizon's alarm belongs in a deadline tier rather than on the pager.
  • How do you stop the day-ahead deadline ticket from being quietly ignored?
    Give it an expiry that escalates rather than closes. The item carries a hard time - a fixed lead before gate closure - at which it becomes a page if nobody has touched it, plus a named owner for that horizon. Without escalation the middle tier degrades into a suppression mechanism, and the team discovers the model was bad only when the submitted schedule turns out wrong.

saying these in an interview costs you the question

  • Tiers alarms by error magnitude instead of the decision deadline
  • Pages at 03:00 for a submission that closes eight hours later
  • Files a deadline ticket with no expiry and no owner
  • Assumes a rollback repairs forecasts already submitted
  • Uses one severity for every horizon because one model produces them
  • Suppresses all overnight pages to protect the rotation