skip to content

Why do idle OpenSSH sessions through a NAT gateway or firewall freeze and drop, and how does ServerAliveInterval differ from TCPKeepAlive?

level: middleimportance: should knowfreq 27%

answer

  1. middleboxes forget idle flows
  2. probe inside the encrypted channel
  3. interval times count
  4. kernel probes are spoofable

basics

~20 s

Stateful middleboxes drop idle TCP flows, so the next keystroke goes nowhere. ServerAliveInterval sends an encrypted request inside the SSH connection every N idle seconds and disconnects after ServerAliveCountMax (3) unanswered; TCPKeepAlive uses spoofable kernel probes.

solid answer

~50 s

A NAT gateway or stateful firewall keeps a table entry per flow and removes it after a period of silence; when the session resumes, packets no longer match and the terminal hangs until TCP gives up. `ServerAliveInterval` makes OpenSSH's client send a request **inside the encrypted SSH connection** whenever nothing has been received for that many seconds; the traffic keeps the middlebox entry alive, and after `ServerAliveCountMax` unanswered requests (default 3) ssh disconnects, so `ServerAliveInterval 15` detects a dead server in about 45 seconds. `ServerAliveInterval` defaults to 0, meaning off. `TCPKeepAlive`, default `yes`, relies on the operating system's TCP keepalive probes: they are not authenticated, so the manual calls them spoofable, and their timer is set by the system, often far longer than a middlebox's idle timeout. The server has the mirror pair, `ClientAliveInterval` and `ClientAliveCountMax`.

go deeper

for a junior

Recall that idle sessions die at NAT gateways and firewalls and that ServerAliveInterval keeps them alive.

for a middle

Explain interval times count, why server alive messages travel encrypted, and why TCPKeepAlive is spoofable and set by the operating system.

for a senior

Show how you choose different interval and count values for interactive users, automation and long-lived tunnels on the same fleet.

for a principal

Weigh client-side keepalives in a shared config against server-side ClientAlive settings and middlebox timeouts you do not control.

## The symptom and its cause An SSH session left idle over lunch is frozen on return: keystrokes do nothing, and after a long wait the client reports a broken pipe or a reset. The usual cause is a device in the path that tracks connections, such as a **NAT gateway** or a **stateful firewall**. It keeps an entry per TCP flow and deletes it after a period of inactivity. When the client sends again, the packet matches no entry; it is dropped or answered with a reset, and the session is dead even though both endpoints are fine. How address translation keeps that state is its own subject; for the SSH client the point is that **silence ends the session**. ## Two kinds of keepalive OpenSSH's `ssh_config(5)` stresses that server alive messages are **very different** from `TCPKeepAlive`: | Property | `ServerAliveInterval` / `ServerAliveCountMax` | `TCPKeepAlive` | |---|---|---| | Layer | SSH protocol message through the encrypted channel | operating system TCP keepalive probes | | Default | interval 0 (off), count 3 | `yes` | | Spoofable | no, protected by the SSH transport | yes, plain TCP segments | | Timing | set per host in seconds | set by the operating system, often hours | | Detects | an unresponsive server or a dead path | a crashed peer or a dead route | **ServerAliveInterval** sets how many seconds of receiving nothing from the server pass before ssh sends a request through the encrypted channel asking for a response. **ServerAliveCountMax** sets how many such requests may go unanswered before ssh disconnects. With `ServerAliveInterval 15` and the default count of 3, an unresponsive server is dropped after roughly 45 seconds. **TCPKeepAlive** asks the system to send TCP keepalive probes. The manual notes the trade-off: a crashed peer or dead route is noticed, but the connection also dies if the route is down only temporarily. Because the probes are ordinary TCP segments, an attacker in the path can forge them and keep a dead connection looking alive, which is why the manual calls this option spoofable. ## A setting that works through middleboxes ```ini Host *.internal.example ServerAliveInterval 30 ServerAliveCountMax 4 ``` Choose the interval below the shortest idle timeout on the path, so the traffic refreshes the middlebox entry. Then decide how long a real outage may last before you want the session dropped: interval times count, here about two minutes. - **Interactive users** on flaky networks prefer a larger count, so a short outage does not kill the session. - **Automation** prefers a short total, so a hung host fails the job quickly instead of stalling a whole run. - **Long-lived forwards** such as a background tunnel need it most, because they can sit idle for hours. ## The server's side and the order of checks The same idea exists on the server as `ClientAliveInterval` and `ClientAliveCountMax` in `sshd_config`, set by the server's administrator and checked from the other end. Either side's probes keep the middlebox entry alive. For a client you control, the client-side pair is the one you can change today. Use `ssh -G host | grep serveralive` to confirm which values apply to a given host, since the first-value rule means a `Host *` block higher in the file may already have set them. ## Telling the failures apart - A session that freezes after a fixed idle period, every time, on one network path points at a middlebox timeout; server alive messages fix it. - A session that ends with ssh reporting that the server did not answer its alive requests means the configured count was reached: the path or the server really stopped responding. - A session closed by the server after a set idle time, with the client still reachable, points at an idle policy set by the server's administrator rather than at a middlebox; ask before working around it.

  • With `ServerAliveInterval 30` and `ServerAliveCountMax 4`, roughly how long does an OpenSSH client wait before dropping a dead server?
    About 120 seconds: a request goes out after each 30 seconds of silence, and the client disconnects once four of them go unanswered. Raising the count tolerates longer outages; lowering it fails faster.
  • Why might you set `TCPKeepAlive no` while keeping server alive messages on?
    Kernel TCP keepalives can drop a connection during a brief route outage and can be forged by someone in the path. Server alive messages travel inside the encrypted connection, so they cannot be forged, and their interval and count are set explicitly per host.

  • TCPKeepAlive is off by default in the OpenSSH client
  • ServerAliveInterval only matters on the server side
  • Server alive messages are plain TCP probes like TCPKeepAlive
  • ServerAliveInterval 15 drops the server after 15 seconds
  • Kernel TCP keepalives cannot be spoofed because the session is encrypted