skip to content

Sometimes you genuinely must interpolate a value into a command line that a shell will parse. Explain why quoting and escaping is a structurally weaker defence here, and what exactly you have to model to get it right.

level: seniorimportance: should knowfreq 40%

answer

  1. escaping = reimplementing someone else's lexer
  2. single quotes can't contain a single quote — close/escape/reopen
  3. double quotes still expand $ and backticks
  4. Windows: different escape char, parse-time expansion, callee re-splits
  5. splitting + globbing happen AFTER substitution

basics

~20 s

Escaping means reimplementing a specific shell's lexer. Shells diverge on which quoting context suppresses what, on escape characters, and on how many times a string is parsed — so the escape is only correct for the exact interpreter you actually invoke, which you often do not control.

solid answer

~1 min

Escaping sits below structural separation because it is a *model* of an interpreter, and it is only as correct as the model. To quote correctly you must know precisely which lexer will run and how it behaves along three axes. **Which context suppresses what.** In POSIX shells single quotes suppress everything — and cannot contain a single quote at all, so the escape is a close-reopen trick — while double quotes still honour `$`, backticks and backslash. **What the escape character is, and how many passes there are.** The Windows command interpreter uses a different escape character, expands environment references *at parse time*, and has a delayed-expansion mode that re-parses after substitution; the invoked program then re-splits the same string with its own rules, so one string is lexed twice by two grammars. **What happens after expansion.** Word splitting on the field separator and pathname expansion run *after* a value is substituted, so an unquoted expansion is unsafe even with no metacharacters at all. So: pin the shell you invoke, use that shell's own quoting routine, keep the escaped region as small as possible — and prefer moving the value into the environment or a file, which needs no quoting.

code

text · 13 lines
text
POSIX sh, double quotes:
  cmd "$name"          name = $(id)      -> command substitution still runs

POSIX sh, single quotes, naive wrapper:
  cmd '<name>'         name = it's       -> quote closes early; rest is syntax

POSIX sh, unquoted, no metacharacters at all:
  cmd $name            name = "a b"      -> becomes TWO arguments (word splitting)
                       name = "*"        -> becomes the directory listing

Windows command interpreter:
  cmd "<name>"         name contains %VAR%  -> expanded at parse time,
                                              then the callee re-splits the line

go deeper

for a junior

Know that quoting is error-prone and that the preferred answer is to avoid the shell; be able to say that double quotes do not stop command substitution.

for a middle

Explain that escaping means modelling a specific shell's grammar, give the single-quote-inside-single-quotes problem, and note that splitting and globbing occur after substitution.

for a senior

Name concrete divergences across at least two or three interpreters, explain multi-pass parsing and callee re-splitting, and give the practical ordering: pin the shell, use its own quoter, move the value out of the command line entirely.

for a principal

Frame escaping as an accepted-risk rung with a documented interpreter assumption, and design so that only a small audited number of call sites are allowed to occupy it at all.

## Why this rung is weaker, stated precisely Structural separation removes the interpreter from the path. Escaping keeps the interpreter and instead transforms the data so that the interpreter will not read it as syntax. That transformation is a **model of the interpreter's grammar**, written by you, and it is correct only to the extent that the model matches the interpreter that actually runs. Three things can invalidate it: you were wrong about the grammar, you were wrong about *which* interpreter runs, or the string gets parsed more than once. All three happen routinely. This is the same reason manual escaping is discouraged at every other injection sink; what is specific here is how many independent grammars are in play. ## Axis one: quoting contexts differ, including in what they cannot express In a POSIX-family shell, single quotes are absolute — every character between them is literal — with one crucial exception: a single quote itself cannot appear inside single quotes, at all, by any escape. The standard technique is to end the quoted run, emit an escaped quote outside it, and start a new run. Any hand-rolled "wrap it in single quotes" helper that does not handle this is broken by exactly one character. Double quotes are not absolute: parameter expansion, command substitution, arithmetic expansion and backslash escapes all still occur inside them. So a value containing `$(...)` inside double quotes still executes a command. Some shells add further quoting forms with their own rules for escape sequences inside them. ## Axis two: escape characters and multi-pass parsing The Windows command interpreter is a different language, not a dialect. Its escape character is different, and it expands environment-variable references *while parsing the line*, before your escaping had any effect on the resulting text — with a delayed-expansion mode that substitutes at execution time and re-parses the result, so a value can be introduced into the line after every check you performed. Then the target program receives the raw command line and splits it into arguments with its own convention. Net effect: one string is lexed by two grammars in sequence, and an escape that satisfies the first can be undone or re-interpreted by the second. Other interpreters add their own passes. A command shell built around an object pipeline tokenizes with its own rules and then *re-serialises* arguments into a native command line when calling an external program, so a value that was safely quoted in that shell's own syntax can be re-split on the way out. ## Axis three: what happens after substitution In POSIX shells, expansion is not the last step. After a variable is substituted, the result undergoes **word splitting** on the characters in the field-separator variable and then **pathname expansion**. This is why an unquoted expansion is unsafe even when the value contains nothing that looks dangerous: a space turns one argument into two, an asterisk turns it into a directory query, and an attacker who can influence the field-separator variable changes what counts as a boundary. Correct quoting has to survive this stage, which is precisely the stage people forget because it happens *after* the text they were looking at. ## Which shell is "the shell"? Escaping correctly requires knowing the interpreter, and often you do not. The system shell may be a small POSIX shell or a large feature-rich one, and they differ in extensions. A remote-execution path runs the *account's login shell*, which the account owner may have changed. A build tool may use a configurable shell. A container image may not contain the shell you assumed. If your escape is tuned to one grammar and a different one runs, the transformation is simply wrong. ## How to do it when you must When the shell genuinely cannot be removed — an existing script you are calling, a scheduled entry, a remote command — apply these in order: 1. **Pin the interpreter.** Invoke a specific shell binary explicitly rather than "whatever the default is," so your model has a fixed target. 2. **Use that shell's own quoting routine**, from a maintained library, rather than a hand-written wrapper. This is the one place where reusing someone else's carefully-tested lexer model is strictly better than writing your own. 3. **Minimise the escaped region.** Get the untrusted value out of the parsed text entirely wherever possible: pass it in the child's environment (the environment is an array of strings, not a parsed line), or write it to a file and pass the path, or feed it on standard input. A value that never appears in the command line does not need quoting. 4. **Prefer closed-world enumeration for anything structural.** If the user chooses *which* command, *which* flag, or *which* target, do not interpolate their string — map a public token to a code-owned invocation. A pattern that constrains characters is open-world and still admits every legitimate-looking flag you did not think about; a finite map is closed-world and owned by your code. This is why enumeration takes the top rung whenever structure itself must vary. 5. **Assume the parse happens again.** Ask what the callee does with the argument: does it re-invoke a shell, expand its own configuration, or interpret the value as an option? Escaping the outer grammar does nothing about the inner one. ## The honest summary for an interview Escaping is a legitimate rung — it is not "never do this" — but it is a *heuristic that depends on a model*, whereas an argument-vector launch is a guarantee that depends on nothing. Say that explicitly, name at least two grammars that diverge, and mention that word splitting and pathname expansion occur after substitution. That answer distinguishes someone who has debugged this from someone who has read a checklist.

  • If you wrap every interpolated value in single quotes in a POSIX shell, what is left to get wrong?
    The single quote itself, which cannot appear inside a single-quoted run under any escape — you must close the run, emit an escaped quote, and reopen, and a wrapper that misses this is broken by exactly one input character. Beyond that, the quoting only protects that one word: the command name, the flags, and the structure around it are still your responsibility, a leading dash inside the quotes is still parsed as an option by the callee, and if the callee re-invokes a shell your quoting has already been consumed.
  • Why is it safer to pass an untrusted value in the child's environment or on standard input than in the command line?
    Because neither is a parsed line. The environment is delivered as an array of strings, and standard input is a byte stream the program reads as data, so no lexer converts your value into syntax. It also removes the value from process listings, which command-line arguments are visible in. The remaining caution is to build the environment explicitly rather than letting an untrusted value name a variable that changes loader or shell behaviour.
  • Where does closed-world enumeration take the top rung instead of structural separation?
    Whenever the *structure itself* is what varies with user input — which program to run, which flags to include, which subcommand. You cannot pass a flag as a bound value, because it is syntax by definition. So you map a public token to a code-owned invocation: the set of allowed commands and flags is finite and lives in your source. The reason this beats a pattern is open-world versus closed-world — a character-class rule still admits every dangerous option that happens to match it, whereas an enumerated map admits only what you wrote down.

Escaping is translating a sentence so that no word in it can be mistaken for a command — but you must know exactly which language the listener speaks, and here the sentence is often heard twice, by two listeners speaking different languages.

saying these in an interview costs you the question

  • Writing a quoting helper by hand instead of using the target shell's own routine.
  • Assuming double quotes make a value literal in a POSIX shell.
  • Forgetting that word splitting and pathname expansion happen after a variable is substituted.
  • Escaping for one shell while the code path actually invokes a different one — a login shell, a build tool's shell, or a command interpreter.
  • Escaping the outer grammar without asking whether the callee re-parses or re-invokes a shell.

context