skip to content

A Node 24 HTTP service behind a load balancer returns intermittent 502s that never appear in its own request logs; which server timeout usually explains this, and how do you fix it?

level: seniorimportance: should knowfreq 50%

answer

  1. the balancer pools upstream connections
  2. who closes the idle socket first
  3. five seconds against sixty
  4. a request lands on a closing socket
  5. keepAliveTimeoutBuffer in 24.6

basics

~10 s

Node's keepAliveTimeout (5 s) is shorter than the balancer's idle timeout, so Node closes reused idle connections just as the balancer sends a request on one. Set server.keepAliveTimeout above the balancer's idle timeout.

solid answer

~50 s

A load balancer keeps a pool of connections to each Node instance and reuses idle ones. Node destroys a kept-alive socket after `server.keepAliveTimeout`, 5000 ms by default, while balancers typically hold idle upstream connections much longer. When traffic is sparse, the balancer picks a connection Node is closing at that moment, the request meets a closed or reset socket, and the balancer answers the client with a 502. Node never parsed that request, so nothing shows in its logs. The fix is to make the balancer close first: set `keepAliveTimeout` a few seconds above the balancer's idle timeout, in `createServer` options or before `listen()`, because the value applies only to new connections. Node 24.6 added `keepAliveTimeoutBuffer` (1000 ms), which keeps the socket a second longer than the advertised `Keep-Alive: timeout`, but it does not help a balancer that ignores that header.

code

javascript · 11 lines
javascript
import { createServer } from 'node:http';

// The balancer drops idle upstream connections after 60 s,
// so Node must keep them open longer and never close first.
const server = createServer({ keepAliveTimeout: 65_000 }, (req, res) => {
  res.end('ok');
});

server.listen(3000, () => {
  console.log(server.keepAliveTimeout, server.keepAliveTimeoutBuffer); // 65000 1000
});

go deeper

for a junior

Know that a load balancer reuses connections to Node, and that Node closes idle kept-alive connections after keepAliveTimeout, five seconds by default.

for a middle

Explain why the side with the shorter idle timer closes first, and why the request that loses the race leaves no trace in the Node service.

for a senior

Diagnose from the evidence: 502s correlated with idle gaps, no Node log lines, success on retry. Fix by setting keepAliveTimeout above every pooling layer's idle timeout at construction.

for a principal

Make the timeout ordering a platform rule written down per layer, client idle below balancer idle below Node keepAliveTimeout, and check it whenever a layer changes.

## The symptom The pattern is recognisable: a small, steady rate of 502 Bad Gateway responses, more of them on quiet routes or at night than at peak, no matching errors in the application's logs, and a balancer log that records the upstream connection as closed or reset before a response arrived. Retrying the same request usually succeeds. ## Two parties, two idle timers HTTP/1.1 keep-alive lets one TCP connection carry many requests. A load balancer exploits this by keeping a **pool** of connections open to each Node instance. Both ends run an idle timer on each pooled connection: | Party | Timer | Typical value | |---|---|---| | Node `http.Server` | `server.keepAliveTimeout` | `5000` ms default | | Load balancer | its upstream idle timeout | often around 60 s, set in the balancer | Whichever timer is shorter decides who closes the connection. With Node's default, Node always closes first. ## The race 1. A request finishes and the connection goes back into the balancer's pool. 2. Five seconds pass with no traffic on that connection. Node's timer fires and Node destroys the socket. 3. In the same instant the balancer, which still considers the connection good for another 55 seconds, sends a new request on it. 4. The request meets a socket that is closing or already gone. The balancer sees a reset or an early close and returns 502 to the client. Node never parsed that request, so no handler ran, no access log line was written, and no error was raised. That is why the 502s are invisible in the service. The race needs an idle gap close to Node's timeout, which is why busy connections rarely hit it and quiet ones do. ## The fix: let the balancer close first Set `server.keepAliveTimeout` above the balancer's idle timeout, with a margin of a few seconds. If the balancer drops idle connections after 60 seconds, a Node value of 65000 ms means the balancer always gives up on a connection before Node does, and a balancer never reuses a connection it has already closed itself. - Pass it as a `createServer` option, or set it before `listen()`. The docs note the socket timeout logic is set up on connection, so changing the property later affects only new connections. - Check every layer that pools upstream connections: a balancer, a proxy sidecar, or an in-house gateway each has its own idle timer. - Raising the value costs a little memory per idle connection; for a balancer's bounded pool that is negligible. ## What Node 24 adds, and what it does not fix Node sends `Connection: keep-alive` with `Keep-Alive: timeout=N`, where N is `keepAliveTimeout` in whole seconds, so a client that honours the header can stop reusing the connection before Node closes it. Since v24.6.0, `server.keepAliveTimeoutBuffer` (default 1000 ms) extends the real socket timeout to `keepAliveTimeout + keepAliveTimeoutBuffer`, which the docs say helps reduce `ECONNRESET` errors for such clients. A balancer that ignores the header and relies on its own idle timer still loses the race, so the buffer is not a substitute for raising `keepAliveTimeout`. ## Other causes of upstream 502s to rule out - **`maxRequestsPerSocket`.** When set (it defaults to 0, no limit), Node marks the response that reaches the limit with `Connection: close`, and requests that still arrive on that socket get `503`, emitting `'dropRequest'`. A balancer that ignores the close header surfaces these. - **Crashes and restarts.** An instance that dies mid-response produces 502s too, but leaves crash logs and restart counts behind. - **Shutdown.** Connections cut while an instance stops look the same from the balancer; how a server drains before exit is covered with graceful shutdown. The giveaway for the keep-alive race is the combination: no trace in Node, correlation with idle periods, and success on retry. ## Confirming the diagnosis 1. Compare the balancer's 502 timestamps with Node's request logs; the failing requests are missing on the Node side. 2. Check the balancer's upstream idle timeout against `server.keepAliveTimeout` on the running build, not the value in a config file. 3. Reproduce it by reusing one connection with idle gaps close to Node's keep-alive timeout and watching for resets. 4. After raising `keepAliveTimeout`, watch the 502 rate fall as old pooled connections age out.

  • Why does Node 24.6's keepAliveTimeoutBuffer not remove these 502s on its own?
    It keeps the socket open `keepAliveTimeoutBuffer` (1000 ms) beyond the `Keep-Alive: timeout` value Node advertises, which helps clients that read that header and stop reusing the connection in time. A balancer that ignores the header uses its own, longer idle timer, so Node still closes first and the race remains. Only raising `keepAliveTimeout` above the balancer's timer ends it.
  • A team raised server.keepAliveTimeout at runtime from a config reload, yet the 502s continued for hours. Why?
    Node applies keep-alive socket timeouts when a connection is set up, so changing the property affects only new connections. The balancer's long-lived pooled connections kept the old 5-second behaviour until they were replaced. Set the value in the `createServer` options or before `listen()`, and expect old connections to age out after a change.

  • Raises the balancer's idle timeout, which only widens the gap
  • Looks for the failing requests in Node's access logs
  • Thinks Node itself generated the 502 status
  • Thinks keepAliveTimeoutBuffer alone removes the race
  • Changes keepAliveTimeout at runtime and expects old sockets to follow
  • Blames handler errors because the status is 5xx

Assumes Node 24: keepAliveTimeout defaults to 5000 ms in every supported release, and since v24.6.0 keepAliveTimeoutBuffer (1000 ms) is added to the real socket timeout.