Timeouts and Stale Sockets

## Why one timeout is never enough "The call took too long" is not one failure. A request passes through distinct phases, each with a different failure mode and a different sensible bound, so mature clients expose several timeouts. Getting them wrong produces either hung threads (too generous) or spurious failures under normal latency variance (too tight). ## Connect timeout Covers name resolution in some libraries and always the TCP handshake: SYN, SYN-ACK, ACK. Symptoms it catches: - host down or unreachable - usually an immediate `ECONNREFUSED` rather than a timeout; - a firewall silently dropping SYNs - here the OS retries with exponential backoff for a very long time (over two minutes on Linux defaults), so without an explicit connect timeout the client hangs; - a server whose accept backlog is full. Within a datacentre a connection should complete in single-digit milliseconds, so 100-500 ms is a reasonable bound; over the public internet, 1-3 s. Because it applies before any application work, it is safe to keep tight - and a connect timeout is the strongest evidence a request never reached the application at all, which matters for retry decisions. ## TLS handshake timeout Some libraries fold this into connect, some expose it separately. It bounds the handshake round trips, which can stall on a slow server or an unresponsive OCSP/CRL fetch. ## Read / socket timeout The most misunderstood setting. It is the maximum time the socket may be idle while the client is waiting for data - typically implemented as `SO_TIMEOUT` or an equivalent per-read deadline. It resets whenever a byte arrives. Consequences: - A 5-second read timeout does **not** guarantee a request completes in 5 seconds. A slow trickle keeps resetting it, and a large legitimate download also keeps resetting it, which is exactly why streaming works. - It cannot distinguish "server thinking" from "network path broken" - both look like silence. Because of that, clients increasingly expose a **response / total timeout** covering the entire exchange. Use the read timeout to detect stalls and the total timeout to enforce a deadline; for streaming or long-polling endpoints, keep the read timeout and raise or disable the total one. ## Write timeout Bounds how long sending the request body may stall, which happens when the server stops reading (a slow consumer or a full receive window). Without it, uploading to a wedged server hangs. ## Idle keep-alive timeout A property of the pool, not of a request: how long a connection with no traffic may sit idle before the client closes it. Servers have their own version (nginx `keepalive_timeout`, many managed load balancers 60 s), and they will close first if theirs is shorter. Keeping the client's idle timeout comfortably **below** the server's is the standard way to reduce the stale-connection race, where the client hands out a connection the server has already begun closing. Set it too low and you throw away warm connections and pay handshakes; too high and you accumulate idle sockets and hit that race. ## Pool acquire timeout When every pooled connection is leased, a new request waits for one. That wait is bounded by its own timeout and must be reported as its own error class - confusing it with a connect timeout sends people to debug the network when the problem is client-side concurrency. ## Putting it together A sane baseline for an internal JSON API: connect 200-500 ms, read 1-3 s, total request timeout aligned with the caller's deadline, write timeout similar to read, pool acquire a few hundred milliseconds, client idle keep-alive comfortably under the server's. Every value should be justified by measured latency (p99.9 plus margin), not copied from a tutorial - and every one of them should be set explicitly, because several widely used clients default to waiting forever.