A deploy job runs `ssh build@host 'mytool --version'` and fails with `zsh: command not found: mytool`, yet logging into that same account interactively and typing the command works. The PATH entry is exported from ~/.zshrc. Why does the remote command miss it, and where does the export actually belong?
answer
- which files a remote command reads
- rc means interactive
- env runs for every zsh
- keep it silent — scp parses stdout
- move the export to ~/.zshenv
basics
~20 ssshd runs a remote command in a non-interactive, non-login zsh, which reads only ~/.zshenv and never ~/.zshrc. Move the PATH export to ~/.zshenv, and keep that file silent and fast because every zsh process runs it.
solid answer
~40 sWhen you `ssh host 'cmd'`, sshd does not give you a login shell — it runs the user's shell with `-c` to execute that one command. That shell is neither interactive nor a login shell, so zsh reads only `/etc/zshenv` and `~/.zshenv`. `~/.zshrc` is skipped, and with it the `PATH` export, which is why the binary is not found even though your own terminal finds it fine. The fix is to move environment that non-interactive processes need — `PATH`, `EDITOR`, tool-specific variables — into `~/.zshenv`, leaving `~/.zshrc` for aliases, prompt, key bindings and completion. That file then carries a discipline: it must stay quiet, because `scp`, `sftp` and `rsync` run through a non-interactive shell and treat unexpected stdout as protocol data, and it must stay fast, because it runs for literally every zsh.
code
bash · 6 lines# ~/.zshenv - read by EVERY zsh: remote commands, scripts, subshells
export PATH="$HOME/.local/bin:$PATH"
export EDITOR=vim
# Interactive-only things belong in ~/.zshrc, never here:
# aliases, prompt, key bindings, completion, plugin managers, bannersgo deeper
Know that running one command over ssh is not the same as logging in, and that ~/.zshrc is not consulted in that case.
Explain that sshd runs the remote command through a non-interactive, non-login shell so only ~/.zshenv applies, and name that file as the place the export belongs.
Demonstrate the diagnosis end to end — reproduce with a one-shot command, print PATH from the remote shell — and the discipline that keeps ~/.zshenv silent, fast and idempotent.
Set the rule for the fleet: what may live in a user's environment file versus what the job definition must pin itself, so automation never depends on an individual's dotfiles.
## What ssh actually starts There are two very different things people call "ssh-ing in". `ssh host` allocates a terminal and starts the user's shell as a **login shell**. zsh then reads `.zshenv`, `.zprofile`, `.zshrc` and `.zlogin` — everything. `ssh host 'mytool --version'` starts the user's shell with `-c 'mytool --version'`. No terminal is allocated, and the shell is not a login shell. zsh reads `/etc/zshenv` and `~/.zshenv`, then executes the command and exits. Nothing else is consulted. So the symptom — works when I log in, fails when the job runs it — is not mysterious at all. Two different shells read two different sets of files. ## Why .zshrc is skipped, and why that is right The `rc` file is the *interactive* configuration. Everything in it assumes a human: aliases you type, a prompt you look at, key bindings, completion, colour. A batch process has no use for any of it, and sourcing several hundred lines of it before every remote command would be pure cost. zsh's file split is what makes that separation possible; the mistake is putting environment in the interactive file. ## The fix Move the export: ```zsh # ~/.zshenv export PATH="$HOME/.local/bin:$PATH" ``` Everything a *process* needs goes here. Everything a *person* needs stays in `~/.zshrc`. Things that look like fixes but are not: - **Re-adding the export to `.zshrc`.** It was never read. - **`ssh host -t 'zsh -lic "mytool"'`.** Forcing a login-interactive shell does make the config load, but you have now made a deploy job depend on someone's interactive dotfiles, prompt and all. It is a workaround, not a fix, and it breaks the day a plugin prints something. - **Assuming ssh will pass your local environment.** It will not, beyond what `sshd` is explicitly configured to accept. The most robust option for automation is to not depend on dotfiles at all: have the job invoke an absolute path, or set the environment in the job definition. Dotfiles are a convenience for humans; a pipeline that breaks when someone edits their shell configuration is badly coupled. ## The discipline .zshenv demands Because `.zshenv` runs for *every* zsh — every script with a zsh shebang, every remote command, every subshell that re-execs the shell — it has rules. **It must be silent.** File-transfer tools run a non-interactive shell on the remote side and read its stdout as protocol data. A cheerful `echo "Welcome back"` in `.zshenv` corrupts that stream, and the failure surfaces as a baffling `scp`/`sftp`/`rsync` error that mentions nothing about your shell. If you must print, guard it: ```zsh if [[ -o interactive ]]; then echo "Welcome back, $USER" fi ``` `[[ -o interactive ]]` tests a shell option; `[[ -t 1 ]]` tests whether stdout is a terminal. Either is better than an unguarded print. **It must be fast.** Anything you put here is paid by every zsh process. A `$(...)` call forks; a network lookup can hang a script that had no reason to touch the network. **It must not assume a terminal.** No key bindings, no prompt, no completion setup, no anything that expects line editing to exist. **It must not assume it runs once.** Naively prepending to `PATH` in `.zshenv` grows the variable each time a nested zsh starts. Either make the append idempotent or set `PATH` absolutely rather than prepending. ## Verifying the diagnosis Ask the remote shell directly rather than reasoning about it: ```zsh ssh build@host 'echo $PATH; echo $ZSH_VERSION; echo $-' ``` That prints the `PATH` the job actually sees, confirms the remote shell really is zsh, and shows the option flags — no `i` among them means non-interactive. Compare with the same command run in your interactive session and the missing entry is obvious. ## The adjacent trap The same class of bug has a variant that `.zshenv` does *not* fix: a scheduled job or a service manager may not run zsh at all. If the thing starting your command uses `/bin/sh`, or execs the binary directly with no shell involved, then no zsh file of any kind is read and the environment has to come from the job's own definition. Check what actually starts the process before you edit dotfiles.
- Someone moves their whole ~/.zshrc into ~/.zshenv so "everything always works". What breaks?Every zsh now pays the full interactive cost — every script, every remote command, every subshell — and anything that prints or expects a terminal misbehaves. File transfers break on stray output, startup latency multiplies, and options meant for line editing get set in contexts where they make no sense. The split exists precisely to prevent this.
- How would you check what the remote shell actually sees, without guessing?Run a one-shot command that reports the environment: `ssh host 'echo $PATH; echo $ZSH_VERSION; echo $-'`. That shows the PATH the job really gets, confirms the remote login shell is zsh, and the absence of `i` in `$-` confirms it is non-interactive. Compare against the same output from your interactive session.
- Why is making the deploy job source the interactive config a poor fix?It couples a pipeline to a person's dotfiles. The job then inherits the prompt, plugins and aliases, and breaks the day someone adds a plugin that prints a banner or blocks on the network. Automation should either call an absolute path or get its environment from the job definition.
- You add a PATH prefix in ~/.zshenv and later notice PATH contains the same directory several times. Why?`.zshenv` runs for every zsh, including nested ones, so a plain prepend re-applies each time a shell re-execs or a subshell starts. Either set PATH to an absolute value in that file, or guard the prepend so it only adds the entry when it is not already present.
saying these in an interview costs you the question
- Adds the export to ~/.zshrc again and retries
- Thinks ssh runs a login shell for a remote command
- Puts an echo or a banner in ~/.zshenv
- Says zsh reads ~/.profile for login shells
- Fixes it with ssh -t and an interactive login shell