In OpenSSH's sshd_config, how does `Subsystem sftp internal-sftp` differ from `Subsystem sftp /usr/libexec/sftp-server`, and when would you choose each?
answer
- a program versus a built-in
- who starts the login shell
- support files inside a chroot
- logging flags on the line
basics
~20 sBoth answer the sftp subsystem request. A sftp-server path makes sshd run a separate program through the user's login shell; internal-sftp runs the same server inside sshd with no binary, libraries or shell, the usual choice for chrooted transfer accounts.
solid answer
~40 s`Subsystem` maps the name a client asks for to what sshd runs. With a path such as `/usr/libexec/sftp-server`, sshd executes that program through the user's login shell, and it speaks SFTP on its stdin and stdout. With the reserved name `internal-sftp`, sshd runs the same server code in-process: nothing is executed, so a chroot needs no binaries or libraries and the account's shell never starts. It takes the same arguments as `sftp-server`, but sshd's `LogLevel` and `SyslogFacility` do not apply to it, so logging needs `-l` and `-f` on the line. Choose the external program on general-purpose hosts or to test a server build; choose `internal-sftp` for chrooted or transfer-only accounts. Note that sshd defines no subsystem by default: the shipped `sshd_config` adds the `sftp-server` line.
code
ini · 5 lines# /etc/ssh/sshd_config, OpenSSH 10.5: keep exactly one Subsystem sftp line
# External program, started through the user's login shell:
#Subsystem sftp /usr/libexec/sftp-server -f AUTH -l INFO
# In-process server; LogLevel and SyslogFacility do not apply to it:
Subsystem sftp internal-sftp -f AUTH -l INFOgo deeper
Recall that the Subsystem line in sshd_config decides what answers SFTP: the sftp-server program or the built-in internal-sftp.
Explain the mechanics: an external program started through the login shell versus an in-process server, the same flags, and why logging needs -l and -f.
Show production judgment: internal-sftp for chrooted or transfer-only accounts, first definition wins, Match overrides since 9.5, and sshd -T before a reload.
Weigh one global SFTP server against per-group overrides, and how that choice shapes logging, confinement and the blast radius of a configuration mistake.
## What the Subsystem directive does When an SSH client sends a `subsystem` channel request (RFC 4254, section 6.5), it names a **service**, not a program. OpenSSH's sshd looks that name up in the `Subsystem` lines of `sshd_config`; each line gives a name and a command with optional arguments. Points worth knowing in OpenSSH 10.5: - **No subsystem is defined by default.** The compiled-in default is empty. The `sshd_config` file shipped with OpenSSH adds `Subsystem sftp /usr/libexec/sftp-server`, and packagers adjust the path. Without such a line, sftp clients fail right after authentication, because sshd answers the request with `SSH_MSG_CHANNEL_FAILURE`. - **The first definition of a name wins.** For each keyword sshd uses the first value it reads, and a second `Subsystem sftp` line is ignored, so an "override" appended to the end of the file does nothing. - **A `Match` block can override `Subsystem` since OpenSSH 9.5**, so one group of users can get different arguments or a different server. - **The name is just a name.** `sftp` is the one SFTP clients ask for; other services start the same way under their own names. ## The external program: sftp-server `sftp-server` is a standalone program that, in its manual's words, speaks the server side of SFTP to stdout and expects client requests from stdin. It is not meant to be run by hand; sshd starts it for a subsystem request like any other remote command, as the authenticated user and **under that user's login shell** from the password database. Three consequences follow: 1. The account needs a shell that will run commands; a shell that refuses every command blocks the subsystem too. 2. Anything the shell's startup files print to standard output lands in the SFTP stream ahead of the server's first packet and breaks the client. 3. Inside a `ChrootDirectory`, the program, its shared libraries and whatever else it touches would all have to exist inside the chroot tree. Its advantage is separation: it is an ordinary process you can replace, wrap with another command, or test on its own (`sftp -D` connects the client straight to a local server program). ## The in-process server: internal-sftp `internal-sftp` is a reserved name, not a path. sshd recognises it and runs the SFTP server code linked into sshd itself (since OpenSSH 4.9) in the session's own process. - **No executable, libraries or support files are needed**, which is why the `sshd_config` manual recommends it with `ChrootDirectory`. - **No login shell is started**, so shell startup output cannot reach the SFTP stream. - **It takes the same arguments as `sftp-server`**: `-d` start directory, `-f` syslog facility, `-l` log level, `-P` and `-p` denied and allowed requests, `-R` read-only, `-u` umask. - **Logging is set on the line.** sshd's `LogLevel` and `SyslogFacility` do not apply to it, and the SFTP server's own default level is `ERROR`, so file transactions are logged only with `-l INFO` or `-l VERBOSE`. ## Seeing the request from the client side - **`sftp`** sends the `subsystem` request for `sftp` for you; `sftp -s name` asks for a different subsystem name. - **`sftp -s /usr/libexec/sftp-server`** gives a path instead of a name, so the client runs that program as a remote command, useful when the server defines no subsystem. - **`ssh -s [email protected] sftp`** sends the same subsystem request by hand; `SessionType subsystem` is the `ssh_config` equivalent. How a chrooted, shell-less transfer group is built on top of this (`Match`, `ChrootDirectory`, `ForceCommand internal-sftp`) is the subject of SFTP hardening; the point here is why that recipe names `internal-sftp`. ## Choosing between them | Question | `Subsystem sftp /usr/libexec/sftp-server` | `Subsystem sftp internal-sftp` | |---|---|---| | What runs | a separate program, via the login shell | server code inside sshd | | Chroot needs | binary, libraries and support files | nothing | | Shell startup output | can corrupt the stream | not involved | | Logging | its own `-l` and `-f` (defaults `ERROR`, `AUTH`) | the same flags; `LogLevel` does not apply | | Typical use | general-purpose hosts, testing a server build | transfer-only and chrooted accounts | A practical rule: keep exactly one `Subsystem sftp` line, decide which server it names, put the logging flags on it, and confirm what sshd will really use with `sshd -T`, which prints the effective configuration, before you reload.
- An OpenSSH server has no `Subsystem sftp` line and you cannot change its config; can a client still use SFTP?Only if an SFTP server binary exists on the host and the account may run commands. `sftp -s /usr/libexec/sftp-server` gives a path instead of a subsystem name, so the client runs that program as a remote command and speaks SFTP to it. Otherwise the subsystem request simply fails after authentication.
- You append `Subsystem sftp internal-sftp -l INFO` to the end of `sshd_config`, yet nothing changes; why?For each keyword sshd uses the first value it reads, and a duplicate `Subsystem sftp` is ignored, so the earlier `sftp-server` line still applies at its default `ERROR` level. Edit or replace the original line, or override it in a `Match` block (possible since OpenSSH 9.5), and check the effective result with `sshd -T`.
- internal-sftp is a binary you must copy into the chroot
- internal-sftp follows sshd's LogLevel automatically
- sshd serves SFTP even when no Subsystem line exists
- A later Subsystem line overrides an earlier one of the same name
- internal-sftp speaks a newer SFTP version than sftp-server
Assumes OpenSSH 10.5; a Match block may override Subsystem since 9.5, and no subsystem is defined unless sshd_config adds one.