skip to content

In a comfort model's serving path, what should happen to a request whose feature is already past its declared age bound?

level: seniorimportance: should knowfreq 46%

answer

  1. flag, fall back, or refuse
  2. never serve silently
  3. each rung checks its own bound
  4. cost of wrong against cost of none
  5. measure the degraded rate per region

basics

~20 s

Three defensible responses: serve the prediction with the staleness recorded, substitute a coarser feature that is still inside its own bound, or refuse to score and let the caller fall back. Serving silently is the one wrong answer.

solid answer

~50 s

Once the read path can see the age, a breach becomes a decision rather than an accident. Order the options as a ladder: use the feature if it is inside its bound; otherwise use a coarser substitute — the hourly mean indoor temperature in place of the current reading — provided that substitute is inside *its own* bound; otherwise return no prediction and let the caller apply its own default. Which rung you stop at is set by comparing the cost of a wrong answer against the cost of no answer for that consumer: a slightly wrong comfort score nudges a setpoint and is cheap, so falling back is right; an action that spends money or locks a device is not, so refusing is right. Whatever fires, the response carries a degraded flag and the rate of each rung is measured.

code

pseudocode · 13 lines
pseudocode
value = read(onlineStore, "indoor_temperature_now", homeId)

if now - value.writtenAt <= 60 seconds:
    return score(model, value, degraded = false)

coarse = read(onlineStore, "indoor_temperature_hourly_mean", homeId)

if now - coarse.writtenAt <= 3600 seconds:
    count("fallback_used", feature = "indoor_temperature_now")
    return score(model, coarse, degraded = true)

count("refused", feature = "indoor_temperature_now")
return no_prediction

go deeper

for a junior

Know that a breach of the age bound needs a defined response, and that the three available ones are marking the prediction, substituting a coarser feature, or returning nothing at all.

for a middle

Explain the ladder's order and why each rung must check its own bound, so a fallback does not replace one known stale value with another.

for a senior

Make the choice from the consumer's downstream action — reversible nudge versus costly commitment — and show how you would monitor the degraded rate by region rather than by request.

for a principal

Decide what the platform's default should be across many consuming teams, knowing a serve-flagged default hides problems and a refuse default makes every freshness incident an outage.

## The three responses, and what each one means When a value is past its declared age bound, the serving path has exactly three honest moves: - **Serve flagged.** Score on the stale value anyway, but mark the prediction as degraded and count it. The caller knows what it received. - **Fall back to a coarser feature.** Substitute something with a looser bound that is still being met — an hourly mean where the sixty-second reading has stopped, a population default where a per-home value is unavailable. The prediction is worse but honest, and still marked. - **Refuse to score.** Return no prediction and let the caller apply whatever it does when the model is unavailable — usually a schedule or a fixed rule. The fourth option, serving silently, is the one the whole freshness apparatus exists to remove. It is not a policy; it is the absence of one. ## Choosing between them The decision is a comparison of two costs, not a matter of taste: | question | pushes toward | |---|---| | Is a slightly wrong answer cheap for the caller? | serve flagged or fall back | | Does the prediction trigger an irreversible or costly action? | refuse | | Does the caller already have a sensible default of its own? | refuse, and let it use the default | | Is a coarser version of the same signal available and inside its bound? | fall back | | Would a refusal take the whole product offline for this user? | serve flagged | In the comfort platform the prediction nudges a setpoint by a degree or two. A coarse estimate is far better than a thermostat that stops responding, so the ladder falls back and only refuses when even the coarse feature is stale. ## Rules the ladder must obey 1. **Every rung checks its own bound.** The classic broken fallback substitutes a feature that is itself out of date, which converts one known breach into two unknown ones. 2. **Nothing is silent.** The degraded state travels in the response so the caller can treat the prediction differently, and it is counted so the rate is visible. 3. **The offline imputation default is not the serving fallback.** The value a training pipeline substitutes for a missing row and the value the serving path uses when a feature is stale are different decisions made by different people for different reasons; reaching for the first because it is already coded is how a fallback becomes invisible. 4. **The degraded rate is a monitored number.** A path that fires for 0.1% of requests is a safety net; the same path firing for 30% is the system running permanently in its degraded mode while every dashboard stays green. ## Why the degraded rate matters more than the individual breach A single stale read is uninteresting. What the design has to surface is the **share** of traffic taking each rung, broken down by region or partition, because staleness is almost always partition-shaped: one ingestion path stops and a definable set of homes goes stale together. A degraded rate that jumps from a fraction of a percent to double digits for one region is a precise, actionable signal that no request-level error rate would ever produce — because, by construction, none of those requests failed. ## What this does not solve A fallback ladder protects the live request. It does nothing for the history: while the feature was stale, the values written into the offline series for those homes were frozen or absent, so any training data assembled over that window is affected. Handling the live request and repairing the record are two separate pieces of work, and the second one is a backfill.

  • How do you decide whether to fall back or to refuse?
    Compare what the caller does with each outcome. If it nudges a setpoint, a coarse estimate beats no estimate and falling back wins. If it commits money, locks a device or is hard to reverse, a wrong answer costs more than an absent one and refusing wins. The question is about the consumer's downstream action, not about how confident the model feels.
  • What should the degraded flag reach besides the caller?
    The freshness alarm, as a rate rather than an event. Counting each rung of the ladder by feature and by region turns a silent condition into a measurable one, and it distinguishes a safety net that fires rarely from a system quietly running in permanent fallback — a state that produces no errors at all.

saying these in an interview costs you the question

  • Serves the stale value and records nothing about having done so
  • Treats refusing to score as always safer than answering
  • Falls back to a substitute feature that is itself past its bound
  • Substitutes the offline imputation default and marks nothing
  • Assumes the caller can spot a degraded prediction without a flag
  • Watches individual breaches instead of the degraded rate per region