skip to content

Why did OpenSSH 9.8 and 10.0 split sshd into sshd, sshd-session and sshd-auth, and what does that split change for someone hardening a server?

level: seniorimportance: nice to knowfreq 8%

answer

  1. privilege separation, now three binaries
  2. the listener never parses network data
  3. pre-auth code in its own address space
  4. parents watch children for crashes
  5. log lines change their tag

basics

~20 s

It is privilege separation made structural: the root listener only accepts connections, sshd-session is each connection's privileged monitor, and since 10.0 a sandboxed, unprivileged sshd-auth handles pre-authentication traffic in its own address space. Operators see new log tags and binaries.

solid answer

~50 s

OpenSSH has long used privilege separation; 9.8 and 10.0 turned it into three binaries. `sshd`, the listener, keeps root but only loads the configuration, accepts connections and tracks the pre-authentication phase to enforce `MaxStartups` and `PerSourcePenalties`; it never handles network-derived data. For each connection it executes `sshd-session` (split out in 9.8), the privileged monitor. Since 10.0 that monitor executes `sshd-auth`, which drops privilege, sandboxes or chroots itself (by default into `/var/empty`) and runs key exchange and user authentication, asking the monitor over RPC for privileged steps such as checking a password or signing with the host key. After login a `sshd-session` child running as the user carries the session. For hardening: pre-authentication bugs land in an unprivileged, disjoint address space; privilege separation can no longer be disabled (9.8); log lines carry the new process names; packages must ship the new binaries.

go deeper

for a junior

Recall that sshd separates privileges: a root listener, a per-connection privileged monitor, and an unprivileged process that talks to the network before login.

for a middle

Explain which binary does what, the listener, sshd-session and sshd-auth since 10.0, and how privileged steps cross from sshd-auth to the monitor by RPC.

for a senior

Translate the design into operations: update log filters for the new process tags, make sure packages ship sshd-auth, keep /var/empty and the privsep account clean, and patch promptly.

for a principal

Use the design to argue exposure and patch cadence: structural isolation lowers, but does not remove, pre-authentication risk, so an internet-facing sshd still needs a fast upgrade path.

## Privilege separation in one paragraph **Privilege separation** (privsep) means the operations that need root run in a small privileged **monitor** process, while the code that parses untrusted network data runs unprivileged and, where the OS allows it, sandboxed. OpenSSH's own README.privsep lists the goals: shrink the privileged attack surface, make the most exposed code easy to sandbox, limit information shared between privileged and unprivileged code, and contain a successful attack inside the unprivileged process. OpenSSH has done this for many years; what changed in 9.8 and 10.0 is that the pieces became **separate executables**. ## The three binaries | Binary | Runs as | Job | Since | |---|---|---|---| | `sshd` (listener) | root | loads and checks the configuration, accepts connections, tracks each connection through pre-authentication to enforce `MaxStartups` and `PerSourcePenalties`; never handles network-derived data | the slimmed listener since 9.8 | | `sshd-session` | root (the monitor); its post-login child runs as the user | per-connection privileged monitor, performing privileged operations on request; after login, also the user-privileged process carrying the session | 9.8 | | `sshd-auth` | an unprivileged account (`sshd` by default on most platforms), sandboxed and chrooted | speaks the SSH protocol on the network socket during key exchange and user authentication, then exports its state and exits | 10.0 | ## A connection's life ```pseudocode listener sshd (root) accepts TCP connection fork+exec sshd-session # privileged monitor for this connection fork+exec sshd-auth # drops privilege, sandbox, chroot /var/empty key exchange, user authentication over the socket privileged steps -> RPC to monitor (account lookup, password check, host-key signature) on success: serialise protocol state to monitor, exit monitor detaches from listener, forks a child as the user child imports state, runs channels, starts the shell monitor stays privileged for PTY allocation and re-key signatures ``` ## What it changes for hardening - **Smaller pre-authentication blast radius.** The code an anonymous client can reach before logging in now lives in its own unprivileged, sandboxed process with an address space disjoint from the rest of the connection's code; the 10.0 release notes give exactly that as the reason. A memory-safety bug there hands an attacker an unprivileged, chrooted process rather than root. - **Crash detection feeds penalties.** Parents monitor their children for abnormal exits, and sshd uses those conditions for `PerSourcePenalties`: a client that crashes a pre-authentication process is refused for 90 seconds by default, which slows anyone probing for bugs. - **Privsep is no longer optional.** 9.8 removed support for disabling privilege separation and for disabling sshd's re-execution, so there is no switch left to weaken. - **Log lines changed.** Since 9.8 many messages are tagged `sshd-session` rather than `sshd`; since 10.0 some come from `sshd-auth`. Log filters and alert rules keyed to the old tag must be updated. - **Packaging.** The 10.0 release notes tell distributors they must package the new `sshd-auth` binary; a hand-built or copied installation needs all three binaries, not just `sshd`. - **The privsep account and directory.** The unprivileged account should be used by no other service, own no files, have a locked password and a non-login shell; the chroot directory, `/var/empty` by default, should be empty, root-owned and not group- or world-writable. ## Seeing it on a running host The process titles make the structure visible. During pre-authentication a process list shows the listener with its startup counter (for example `sshd: /usr/sbin/sshd -D [listener] 1 of 10-100 startups`, which is `MaxStartups` at work), a root `sshd-session: alice [priv]` monitor, and an `sshd-auth: alice [net]` process running as the unprivileged account. After login the `[net]` process is gone; the root `[priv]` monitor remains, with an `sshd-session: alice@pts/1` child running as alice and her shell beneath it. Since 10.5 the post-authentication monitor also sets its process title so it is easy to identify. In the intended design, no root process parses client traffic before login. ## What it does not change - The `sshd_config` keywords you set are the same; the split is invisible in day-to-day configuration. - It is no substitute for `PermitRootLogin no`, key-only authentication or allow lists: those decide who may log in, whereas privsep limits what a bug can do. - It does not make patching optional. 9.8 itself fixed a critical race condition in sshd, present in portable releases 8.5p1 to 9.7p1, that could allow code execution as root. Structural isolation lowers pre-authentication risk; a prompt upgrade path is still the first hardening control.

  • How does sshd know a client crashed one of its processes, so the crash penalty applies?
    Parent processes monitor their children for abnormal exits, such as crashes or signals. When a pre-authentication process dies that way, sshd records the `crash` penalty, 90 seconds by default, against the client's source address range, which slows anyone repeatedly probing for a memory-safety bug.
  • What should the privilege-separation account and chroot directory look like?
    The unprivileged account `sshd-auth` switches to, `sshd` by default on most platforms, should be used by no other service, own no files, and have a locked password and a shell that denies access. The chroot directory, `/var/empty` by default, should be empty, root-owned and not group- or world-writable.

  • sshd-session is an alternative SSH server you can run instead of sshd.
  • Privilege separation can still be switched off for debugging.
  • sshd-auth runs as root because it has to check passwords itself.
  • The listener sshd parses every client's protocol traffic.
  • With privilege separation, patching sshd promptly no longer matters.

Assumes OpenSSH 10.5: sshd-session was split from the listener in 9.8, and sshd-auth took over pre-authentication in 10.0.