After a macOS major upgrade, a developer's `git` and `cc` commands start failing with `xcrun: error: invalid active developer path`. What are /usr/bin/git and /usr/bin/cc actually doing on macOS, and how do you fix and pin that toolchain?

## The shims The surprise for anyone arriving from Linux is that macOS does not ship a compiler or a git in the usual sense. What is in `/usr/bin` are small stub programs. When you run `/usr/bin/git`, the stub asks `xcrun` to find the real `git` inside the **active developer directory**, and then executes it. If no developer directory is configured, the stub triggers the graphical "install the command line developer tools?" prompt — and in a non-interactive shell, an SSH session or a CI job, that becomes a hard error instead. The active developer directory is one of two things: - `/Library/Developer/CommandLineTools` — the standalone Command Line Tools package: clang, git, make, headers and the macOS SDK. - `/Applications/Xcode.app/Contents/Developer` — the full Xcode, which adds `xcodebuild`, other platform SDKs and the simulators. `xcode-select` reads and writes which one is active, machine-wide. ## Why an upgrade breaks it A major macOS upgrade replaces system content and commonly removes or supersedes the installed Command Line Tools. The stored active-developer path survives the upgrade, now pointing at something that is gone or incomplete, and the shims report exactly that: ``` xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun ``` The message names the path it tried, which is the useful part. Nothing is corrupt; the pointer is stale. ## The repair sequence ```bash xcode-select -p # what is currently active? xcode-select --install # reinstall the standalone Command Line Tools sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer sudo xcode-select --reset # back to the system default xcrun --find clang # prove resolution works xcrun --sdk macosx --show-sdk-path ``` If full Xcode is the target, two more things routinely block a headless machine: the licence must be accepted (`sudo xcodebuild -license accept`) and first-launch components must be installed (`sudo xcodebuild -runFirstLaunch`). Both fail builds with messages that look nothing like "you forgot to accept a licence", so put them in the provisioning script. ## Where the headers went Since macOS Mojave there is no `/usr/include`. System headers live inside the SDK in the developer directory, and the compiler finds them because `xcrun` hands it the right sysroot. This is why an old autotools build that hardcodes `-I/usr/include` fails on a modern Mac, and why `xcrun --show-sdk-path` and the `SDKROOT` environment variable turn up in build scripts. It is also why Homebrew's `brew doctor` complains loudly when the Command Line Tools are missing: any formula that has to build from source needs that SDK. ## Pinning, which is the senior half of the answer `xcode-select --switch` sets a **machine-wide** default. That is fine for a laptop and wrong for a build fleet, because it is global mutable state that any job or any admin can change under you. The per-process override is the environment variable `DEVELOPER_DIR`: ```bash export DEVELOPER_DIR=/Applications/Xcode_16.app/Contents/Developer xcrun --find clang ``` Every tool that resolves through `xcrun` honours it, so a CI job can select its own Xcode without touching the machine and without racing a parallel job that wants a different one. `SDKROOT` pins the SDK the same way. The failure this prevents is subtle and expensive: a hosted macOS runner image is updated, its default Xcode moves a major version, and your builds silently switch compiler and SDK. Nothing errors; the artifacts change. Pinning `DEVELOPER_DIR` per job turns that into a deliberate, reviewable upgrade instead of a surprise. ## The judgement an interviewer is listening for Three things. First, that you know `/usr/bin` on macOS holds shims and not the tools, so "git is installed" and "git works" are different statements. Second, that you read the error's path and treat it as a stale pointer rather than reinstalling everything blind. Third, that machine-wide toolchain state is a liability on shared build machines and the fix is per-process pinning, not a runbook step telling the next person to run `xcode-select --switch` again.