Across a fleet of Go services wired with flags and env vars, which source wins, and when must startup fail?

## What is actually being decided Go gives you no configuration framework, so every service invents its own layering out of `flag`, `os.LookupEnv` and a few `strconv` calls. Left alone, twenty services grow twenty resolution orders and twenty opinions about what a missing value means. The decision is not really technical — any of the plausible orders works — it is about making one choice, implementing it once, and making the outcome visible. ## The precedence order The order almost everyone converges on is **command-line flag, then environment variable, then compiled-in default**. The reasoning is specificity: a flag is something a human deliberately typed for this one invocation; the environment is what the platform injected for this deployment; the compiled default is the fallback that lets `go run ./cmd/server` work on a laptop. In Go this order costs nothing to implement, because a flag's default is an ordinary argument. Resolve the environment first, pass the result as the default, and let `flag.Parse` overwrite it only for flags that actually appeared. When a layer must be applied *after* parsing — a config file whose path is itself a flag — `flag.Visit` tells you which flags were explicitly set so the later layer can skip them. What you should refuse is per-service exceptions. The moment one service inverts the order "because our operators prefer it", every incident involving that service costs an extra ten minutes. ## Fail fast or boot on a default The second decision is sharper, and it is genuinely a trade. **Fail fast** means the process validates every resolved setting and, on any problem, prints to stderr and exits nonzero before it listens, dials a database, or registers with service discovery. The gain is that a misconfigured instance never serves a request: no half-configured process, no requests answered with the wrong feature flags, no data written to the wrong place. The failure is loud, immediate, and attributable to the deploy that caused it. **Boot on a default** means the service starts with a fallback and logs a warning. The gain is availability: a partially broken configuration does not take the service down. The way to hold both is to classify, per setting, rather than to pick one posture for everything: - **Must abort.** Anything where a guess is wrong in a way that is invisible or dangerous: the address of a datastore, a credential, a region or tenant identifier, and anything that changes what the service *means*. There is no safe default for "which database", and a fallback to localhost is how a service quietly writes production data nowhere. - **Must abort even though it parses.** Values that are syntactically fine and semantically impossible: a non-positive request timeout, a rollout percentage above 100, a worker count of zero. Range validation belongs beside type validation. - **May default.** Settings that affect only degree, where the default is documented and logged: log verbosity, cache size, batch size, the port in a development build. ## The organisational cost of fail-fast, and how it is paid The honest objection to fail-fast is real: a config change is usually applied to the whole fleet at once, so a fail-fast service converts one typo into every instance crash-looping, and a crash-looping fleet cannot be rolled back by the same system that just broke it. This is where the decision leaves engineering taste and becomes policy. The mitigations are all outside the process: - **Validate before deploying.** The same resolution and validation code should be runnable as a check in the pipeline against the environment about to be applied. If validation lives in a function that takes an argument slice and an environment lookup rather than reading globals, this is nearly free. - **Roll out progressively and halt on the first failure.** One instance failing to start must stop the rollout rather than proceed to the rest. - **Keep the last-good configuration reachable** so the rollback path does not depend on the thing that broke. - **Make the error report complete.** Collect all validation failures with `errors.Join` and print them together, naming each setting and its offending value. Failing on the first problem forces the engineer bringing up a new environment through one deploy per mistake. A platform team may reasonably overrule a service owner here — for instance requiring that a specific setting have a safe default because the fleet cannot tolerate a simultaneous restart on it. That is a legitimate call, and the service owner's job is to make the consequence explicit rather than to litigate the default. ## Making it stick Ship the policy as a small internal package rather than a document: one function that resolves the layers, one that validates, and a table-driven test over the precedence cases so a future refactor cannot silently reorder them. Require every service to log its effective configuration once at startup with secrets redacted — that line is what turns "it behaves differently in staging" into a ten-second diagnosis. And keep `os.Exit` in `main`. It does not run deferred functions, so a helper that exits on its own skips every cleanup its callers registered and is untestable besides. Helpers return errors; `main` decides to die.