A backend instance that a load balancer had ejected is now passing its health checks again and is about to rejoin the pool. Why can handing it an immediate equal share of traffic break it a second time, and what does a slow-start or warm-up ramp do about that?

## Passing a probe is not the same as being ready for load The health check answers a cheap question: can you respond? Production traffic asks an expensive one: can you respond to a few hundred requests per second right now? Between the two sits everything that a running instance accumulates and that an idle one has lost. **What is cold on a returning instance:** - **Application-level caches** — an in-process cache that normally absorbs most reads is empty, so nearly every request goes to the database. The instance's per-request cost is several times its steady-state cost, and its dependency's load spikes at the same time. - **Connection pools.** Pools to databases, caches and downstream services have been idle-reaped or were never established. The first burst of requests all try to open connections simultaneously, each paying a handshake — and TLS handshakes are expensive enough to matter — while queueing behind a pool whose maximum size throttles them. - **Runtime optimisation state.** On runtimes that compile or optimise while running, code that has not executed recently runs in a slower mode until it is re-optimised, and the optimisation itself costs CPU concurrently with the traffic that triggered it. - **Anything lazily initialised** — class loading, schema or metadata fetches, first-call configuration, a client library that resolves discovery on first use. Each of these makes the first minute of traffic far more expensive per request than the steady state. ## The load-aware balancing trap Here is the part that surprises people. You might expect a returning instance to receive its fair share — one of N. With a load-aware balancing algorithm it receives *more*. Algorithms that pick the backend with the fewest outstanding requests, or the best recently-observed latency, evaluate an idle host as the most attractive host in the pool: zero active requests, no recent slow responses. So they steer a disproportionate burst of new requests to precisely the instance least able to handle them, and they keep doing so until its latency and queue rise enough to make it look ordinary. This is a known, ugly interaction, and it is the strongest single argument for a ramp: the balancer's own heuristics need to be overridden while a host is cold. ## What slow start actually does Slow start (also called warm-up) makes the host's effective weight a function of how long it has been in the pool: near zero at readmission, rising — usually linearly — to its configured weight over a window of tens of seconds to a few minutes. Two properties matter: - It is **time-based, not request-based**, which is what you want: caches and pools warm as a function of both traffic and elapsed time, and a time ramp is predictable and cheap to reason about. - It **overrides the load-aware preference** during the window, so the idle-host burst cannot happen. Sizing the window is empirical: measure how long a cold instance takes to reach steady-state latency and set it there. A JVM service with large caches may need minutes; a small stateless process may need seconds and gain nothing from a ramp at all. ## The two companions **Escalating ejection duration.** If an instance is ejected, readmitted, and ejected again, readmitting it just as quickly the second time produces a flap that perturbs the whole pool. Growing the exclusion period with each consecutive ejection — the ejection duration multiplied by the ejection count is the usual shape — turns a flapping host into a quietly excluded one, and it recovers automatically once the host stays healthy. **The same problem on cold starts.** Nothing about slow start is specific to ejection. A newly started instance, an instance added by autoscaling, and an instance replaced during a rolling update are all cold in exactly the same way. If a deploy shows a latency spike on every new instance, the cause is usually this, and the ramp is the same fix. ## The failure mode you are preventing Without it, the sequence is: eject, recover, readmit at full weight, receive an outsized burst because it looks idle, fall over under cold-start cost, get ejected again — while each cycle redistributes load across the pool and clears whatever locality the balancer had. Slow start converts an oscillation into a monotone recovery, which is why it is a small feature that shows up disproportionately in incident write-ups.