After a team sets a minimum instance count on a latency-critical function, how do you prove it reduced the cold starts users actually wait on?
answer
- init still runs, just earlier
- flag plus gap
- above the minimum, it spills
- matching windows with a deploy and a peak
basics
~20 sMeasure the request-path cold ratio (cold flag set and a near-zero gap since init) and the route's tail in matching before and after windows that each include a deploy and a peak, then check spillover above the minimum and warm-pool utilisation.
solid answer
~40 sI would not look at the init count, because kept-warm instances still initialise, just before traffic. The metric is the request-path cold ratio: invocations with the cold flag set and a near-zero gap between the end of init and the invocation, over all invocations, alongside the route's tail latency. I compare a baseline and a treatment window with the same traffic shape, each containing a deploy and a daily peak, and nothing else changed; a control deployment is better where the platform allows it. Then I read the after picture: cold starts left at peak are spillover above the minimum, cold starts left after deploys mean the new version takes traffic before its warm pool exists, and low utilisation means the minimum is oversized. The report ends with idle cost per avoided cold start.
go deeper
Recall that kept-warm instances still run init, only before traffic, so the init count is not the success metric.
Explain the request-path cold ratio built from the cold flag and the init-to-invocation gap, and why both windows need the same traffic shape.
Show you read the after picture correctly: spillover at peak, deploy waves on the new version, an oversized idle pool, and attribution kept clean by changing one thing.
Frame the result as idle cost per avoided cold start against investing in shorter init, and decide where the latency target justifies standing capacity.
## What the setting changes, and what it does not A **minimum instance count** (on other platforms, **pre-provisioned capacity**) keeps a chosen number of function instances initialised and ready. Requests up to that concurrency find a warm instance. It does not remove init; it moves init off the request path: - **Init still runs.** Kept-warm instances initialise before traffic, so init records, where the platform keeps them, do not drop to zero. - **Concurrency above the minimum spills.** Requests beyond the kept-warm count land on on-demand instances that cold start as before. - **A deploy rebuilds the pool.** The warm instances belong to a version. Whether the platform initialises the new version's instances before traffic moves differs between platforms and settings, so check yours; if traffic moves first, the deploy wave survives the setting. - **Instances still recycle.** Many platforms replace long-lived instances periodically, and each replacement has to initialise again. Because of these four effects, the naive check (did the init count fall?) answers the wrong question. The question is whether users wait on init less often, and by how much. ## Choose the metric before the change Decide what "worked" means before turning the setting on: - **Request-path cold ratio**: invocations whose cold flag is set and whose gap between the end of init and the start of the invocation is near zero, divided by all invocations. A pre-initialised instance has a cold flag but a long gap, so it is excluded. - **Cold penalty**: the cold-minus-warm latency percentile, which the setting is not expected to change; a large shift suggests something else changed too. - **The route's tail latency**, the number the team actually promised. ## Build a fair comparison 1. **Baseline window**: a period with a representative traffic shape that includes at least one deploy and one daily peak. 2. **Treatment window**: the same shape after the change: the same weekday pattern, a deploy, a peak. 3. **Hold other variables still**: no init-trimming release, no traffic-shift change and no memory change inside either window, or the effect cannot be attributed. 4. **Use a control where you can**: where the platform allows it, run two otherwise identical deployments, one with the setting and one without, and compare them over the same hours. ## Read the after picture | Observation | Meaning | Next step | |---|---|---| | request-path cold ratio down at steady load, unchanged at peak | peak concurrency exceeds the minimum and spills | size against the ramp, or accept the peak | | ratio still spikes after each deploy | the new version takes traffic before its warm pool exists | warm the new version before shifting traffic | | warm instances mostly idle | the minimum is far above real concurrency | lower it; idle capacity is billed | | init records unchanged, request-path ratio down | the setting is working as intended | none | Two more checks belong in the report: - **Spillover count**: request-path cold starts that happened while concurrency was above the minimum. It shows how much of the remaining problem the setting cannot reach. - **Utilisation of the warm pool**: average concurrency served by kept-warm instances against the configured count. - **The post-deploy curve per version**: the request-path cold ratio plotted against time since each version took traffic, before and after the change. If the curve is unchanged, the setting is not reaching the deploy wave at all, whatever it does at steady load. ## Turn it into a decision The measurement produces a figure the team can argue about: **the cost of idle warm capacity divided by the request-path cold starts it avoided**. Put that beside the alternative, usually shortening init (trimming dependencies, deferring rare paths), which lowers the penalty for every remaining cold start, including the spillover the setting never reaches. Whether pre-provisioned capacity is worth its cost in general is a trade-off covered with serverless architecture; this measurement is what turns that trade-off into a number for one function. A result to expect: on a function with predictable steady traffic and a sharp but rare peak, the setting removes most cold starts outside the peak and few inside it. That is a correct outcome, not a failed one, as long as the report says which of the two the team was trying to fix.
- After the change, cold starts are gone at steady load but still spike after every release. What do you check?Whether the new version gets its kept-warm instances before traffic moves to it. Warm capacity belongs to a version, and platforms differ on when they initialise it for a new one. If traffic shifts first, the deploy wave survives the setting. Warm the new version before shifting traffic, or shift gradually, and confirm by plotting the request-path cold ratio against time since the version took traffic.
- How do you turn the measurement into a keep-or-drop decision?Divide the cost of idle kept-warm capacity over the period by the request-path cold starts it avoided, and put that beside the cost of shortening init instead. Shorter init helps every remaining cold start, including spillover above the minimum that the setting never reaches. Keep the setting only where the route's latency target needs what init work alone cannot deliver.
- If init records still appear after the change, the minimum instance count is not working.
- Comparing average duration for the week before and after is a fair test.
- A minimum instance count removes cold starts at every level of concurrency.
- Kept-warm instances automatically move to each new version before it takes traffic, on every platform.
- Trimming init and adding warm capacity can be changed together and still be told apart.