skip to content

How do you compute a function's cold-start ratio, and why slice it by deploy and scale-out wave instead of by day?

level: middleimportance: must knowfreq 55%

answer

  1. how often, and how much
  2. waves, not a steady drizzle
  3. new version, new instances
  4. time since the version took traffic

basics

~20 s

Divide cold invocations by all invocations in a window, per function or route. Cold starts arrive in waves after deploys and traffic ramps, so a daily ratio dilutes the burst; slice by version against time since deploy, and by scale event.

solid answer

~40 s

The cold-start ratio is cold invocations over all invocations in a window, using a per-invocation cold label, and I pair it with the cold penalty: the cold latency percentile minus the warm one. A daily figure hides the problem because cold starts come in waves. A deploy moves traffic to a new version that runs on new instances, so right after it most requests can be cold; a traffic ramp adds instances the same way. So I group the ratio by version and plot it against time since that version took traffic, which gives the peak and the decay time, and I read it during scale events rather than across the plateau. Then I alert on the post-deploy window, not the daily number.

go deeper

for a junior

Recall the definition: cold invocations divided by all invocations in a window, which needs a cold or warm label on every invocation.

for a middle

Explain why deploys and traffic ramps create waves of new instances, and how slicing by version and time since deploy exposes them.

for a senior

Show you pair the ratio with the cold penalty, alert on the post-deploy window and decay time, and separate idle-reclaim churn on quiet routes from deploy waves.

for a principal

Weigh which slices become fleet-wide alerts against the noise they add, and set a post-deploy cold-start budget that teams can be held to.

## Two numbers, not one Measuring cold starts in production comes down to two numbers per function or per route: - **Cold-start ratio**: cold invocations divided by all invocations in a window. It says *how often* users meet a cold instance. - **Cold penalty**: a latency percentile for cold invocations minus the same percentile for warm ones. It says *how much* each cold start costs. Both rely on every invocation carrying a cold or warm label (a flag set during init and cleared after the first invocation, plus the platform's own init record where it exists). Their product is a rough measure of user-facing harm. A route with a low ratio and a huge penalty has an **init problem**: make init shorter. A route with a high ratio and a modest penalty has an **instance-churn problem**: make instances live longer or keep some warm. ## Why a daily ratio hides the problem Cold starts are not spread evenly through the day; they arrive in **waves**. A day-long ratio divides a short burst by a full day of warm traffic, and the burst disappears into a small figure. A checkout API whose tail latency jumps for ten minutes after every deploy can report a tiny daily ratio while most requests in those ten minutes are cold. The ratio becomes useful only when it is sliced along the events that cause waves. ## Wave one: the deploy A new version of a function runs on **new instances**. Once the alias or route moves to the new version, the old version's warm instances stop receiving traffic, and the new version starts with none of its own: its pool is built one new instance at a time, each a cold start. The ratio spikes, then decays as the new pool fills. To see it, group the ratio by **version** and plot it against **time since that version began receiving traffic**. Two figures matter: 1. **Peak post-deploy ratio**: how bad the wave gets. 2. **Decay time**: how long until the ratio returns to its steady-state level. For the checkout API, this is the ten-minute window. Shifting traffic gradually spreads the wave over a longer window instead of concentrating it; how to run that shift belongs with deployment operations, not here. ## Wave two: scale-out A traffic step (a marketing email, a backlog of queued messages, the morning ramp) needs more concurrent instances than exist, and each new instance is a cold start. Slice the ratio by **scale event**: overlay concurrent executions per minute and read the ratio *during the ramp*, not across the plateau that follows. How fast a platform may add instances is a separate limit, covered with scaling and throttling; the cold-start view only needs to show that the ramp is where the cold invocations are. ## Quiet churn A low-traffic function or route has a high ratio for a different reason: its instances sit idle long enough to be reclaimed, so many requests find nothing warm. Some platforms also recycle long-lived instances periodically, which adds a trickle of cold starts even under steady load. That is why the ratio should be computed **per route or per function**, never pooled across a busy and a quiet one. ## Building the view | Slice | What it reveals | Usual direction of the fix | |---|---|---| | per version, against time since deploy | the deploy wave and its decay time | shorter init, a gradual traffic shift, warming the new version before it takes traffic | | per scale event | the ramp's share of cold invocations | shorter init, some warm capacity for predictable ramps | | per route at low traffic | idle reclaim | often accept it; keep warm capacity only if the route is latency-critical | | per day | almost nothing actionable | none | A minimal query over structured logs looks like this (the field names are your own tags): ```pseudocode from function_logs where function = "checkout-api" group by version, bucket(minutesSince(versionFirstSeen), 1) select count(where coldStart and gapSinceInit < smallThreshold) / count(*) as coldRatio, percentile(duration, 99) where coldStart as coldP99, percentile(duration, 99) where not coldStart as warmP99 ``` Turn it into a routine: - **Annotate deploys** on the latency dashboard so every wave lines up with its cause. - **Alert on the post-deploy window**, not the daily figure: a peak ratio or a decay time above an agreed budget. - **Track the penalty per version**, because a release that adds a heavy dependency raises the penalty even if the ratio is unchanged. - **Keep the slices in the same units** (per minute since deploy, per minute of ramp) so one release can be compared with the last. The daily ratio still has one use: a sudden change in it is a hint that something structural moved, such as a new traffic pattern or a platform recycling instances more often. Treat it as a smoke alarm, and the sliced ratios as the diagnosis.

  • The post-deploy cold-start ratio of a function is unchanged after a release, but its post-deploy tail got worse. What happened?
    The ratio counts how often requests are cold; it says nothing about how slow each cold start is. A release that added a heavy dependency or an init-time network call raises the cold penalty, so the same share of cold requests now costs more. Compare the init breakdown and the cold penalty between the two versions to find the step that grew.
  • Why can a quiet route show a far higher cold-start ratio than a busy route on the same platform, and should you fix it?
    Its instances sit idle long enough to be reclaimed, so many of its few requests find nothing warm. The absolute number of slow requests is small, so it is often fine to accept. Fix it only if that route is latency-critical, by keeping a small amount of warm capacity or by shortening init so each cold start costs less.
  • What does the decay time after a deploy tell you that the peak ratio does not?
    The peak says how bad the wave is at its worst; the decay time says how long users are exposed. A long decay usually means traffic fills the new version's pool slowly or instances are being replaced repeatedly. It is the number that matches the symptom users describe, such as a tail that stays high for ten minutes after each release.

  • A daily cold-start ratio of a fraction of a percent means cold starts are not a problem.
  • A deploy updates existing warm instances in place, so it causes no cold starts.
  • The cold-start ratio alone shows how much latency cold starts cost users.
  • Pooling a quiet route and a busy route into one ratio gives a fair picture.
  • Cold starts are spread evenly over the day, so any window is representative.