Cold Starts

## What a cold start is A **cold start** is the latency penalty incurred the first time a serverless function instance handles a request, as opposed to a *warm start* where an already-initialized instance handles it. To understand why this penalty exists, you need to understand how serverless platforms like **AWS Lambda**, **Azure Functions**, or **Google Cloud Functions** actually execute code: they don't keep a fleet of instances running for your function at all times. Instead, they run functions inside ephemeral, isolated execution environments — historically containers, and in AWS Lambda's case since 2018 backed by **Firecracker microVMs** — that are created on demand and torn down when unused. ## How an environment gets built The mechanism, step by step: when an invocation arrives and the platform's routing layer finds no idle warm environment for that function, it must - **(1) provision compute** — find capacity on a host, create or restore a microVM/container, allocate CPU/memory; - **(2) bootstrap the runtime** — start the language runtime process, e.g. the JVM, the Node.js V8 engine, the Python interpreter, the .NET CLR; - **(3) initialize the function** — download/mount the deployment package or image, run module-level/static initialization code (imports, singleton construction, DB connection pools, SDK client instantiation, DI wiring); - and only then **(4) invoke the actual handler** with the event payload. Steps 1-3 are the cold-start overhead — pure tax a warm invocation skips because the environment from a previous invocation is frozen and reused, with only step 4 running. ## Why this exists The entire economic and operational proposition of serverless is that you don't pay for or manage idle capacity — the platform multiplexes a shared fleet of hosts across enormous numbers of tenants, spinning environments up only when there's traffic and reclaiming them after a period of inactivity (roughly minutes, undocumented as a hard SLA). This is what lets a rarely-used endpoint cost nothing when idle and still scale to thousands of concurrent invocations during a spike, with no capacity planning. Cold starts are the direct, unavoidable cost of that elasticity: you cannot get *zero idle cost* and *always warm* simultaneously without keeping something running, which is exactly what **provisioned concurrency** (a paid feature) does. ## Upside and downside **Trade-offs.** The upside: - you never provision capacity; - you pay per-invocation/per-ms; - you scale automatically. The downside is unpredictable added latency on some fraction of requests — worse for low-traffic functions and worse the moment traffic scales up faster than existing warm instances can absorb it, since each new concurrent execution needs its own environment. Language and package size matter enormously: | What you ship | Cold-start cost | Why | |---|---|---| | A small **Go** or **Rust** binary | might cold-start in tens of milliseconds | because there's no separate interpreter/VM startup | | A **JVM-based** function with a large dependency graph (Spring, especially) | can take multiple seconds | because interpreted startup, classloading, and reflection-heavy DI wiring are all serial costs paid before application code runs | **Deployment/image size** adds latency too because code must be fetched/mounted before init. ## Where the pain shows up **Failure modes in production.** The most common way this bites teams is a latency SLA violation that only shows up in the tail — p50 looks fine because it's dominated by warm requests, but p99/p999 spikes because a fraction of requests land on a cold environment, especially right after a deploy (every deploy invalidates warm environments) or during a traffic burst (new concurrency always cold-starts). Synchronous request paths behind an **API Gateway** or **ALB** are the most painful because the caller is blocked; async/event-driven consumers are far more tolerant since a few extra hundred milliseconds rarely matters. Teams often *fix* this by - shrinking dependencies; - switching runtime for latency-sensitive paths; - enabling provisioned concurrency; - lazy-initializing expensive resources only when actually needed rather than at module load. ## In the field A concrete scenario: a fintech's payment-authorization API on Lambda, backed by Java/Spring Boot, met p50 SLAs comfortably but periodically breached p99 SLAs during morning traffic ramp-up, when auto-scaling created a burst of new cold environments simultaneously; JVM init plus Spring context startup added roughly 2-4 seconds per cold instance. The team combined provisioned concurrency sized to the morning ramp-up rate with lazy-initializing the heaviest DI-based startup work, cutting the residual cold-start penalty by more than half.