skip to content

Two Go builds of the same commit produce binaries with different SHA-256 hashes. How do you make them identical?

level: seniorimportance: should knowfreq 32%

answer

  1. the artefact describes its own build
  2. diff before theorising
  3. one input was not the same
  4. the checkout directory is an input too
  5. toolchain, target, cgo, deps, flags, paths

basics

~20 s

Compare the build settings recorded in each binary with go version -m, then pin what differs: the toolchain version, GOOS and GOARCH, CGO_ENABLED, the resolved dependency versions, build tags and GOFLAGS. Add -trimpath so the checkout directory stops leaking in.

solid answer

~50 s

Start by reading the artefacts rather than guessing: `go version -m` prints the toolchain version and the build settings baked into each binary, and diffing those two outputs usually names the culprit in one step. The inputs that must match are the toolchain version, the target `GOOS`/`GOARCH`, `CGO_ENABLED` and any C toolchain, the exact resolved dependency versions, the build tags, and anything sitting in `GOFLAGS` on either machine. The one input nobody thinks of is the source directory itself — absolute paths are recorded in the output, so `/home/alice/app` and `/builds/7/app` produce different bytes; `-trimpath` removes that. What you can rule out immediately is the build cache: entries are keyed by every input, so a hit reproduces exactly what a rebuild would, and you can demonstrate that by rebuilding with `-a` and hashing again. If a difference survives all of that, look for nondeterminism in committed generated code or in embedded files.

code

text · 9 lines
text
$ shasum -a 256 app-ci app-laptop

$ go version -m app-ci    > ci.txt
$ go version -m app-laptop > laptop.txt
$ diff ci.txt laptop.txt
< app-ci: go1.27.0
> app-laptop: go1.26.0
<	build	-trimpath=true
>	build	CGO_ENABLED=1

go deeper

for a junior

Know that a Go binary records the toolchain version and build settings that produced it, and that go version -m prints them for inspection.

for a middle

List the inputs that must match for identical output — toolchain, GOOS and GOARCH, CGO_ENABLED, dependency versions, build tags and flags — and explain why -trimpath removes the checkout directory as a variable.

for a senior

Work the diagnosis in order: hash both artefacts, diff their recorded build settings, eliminate the cache by rebuilding without reuse, then compare go build -x output to isolate the differing step.

for a principal

Argue why reproducibility is worth funding — a verifiable artefact rather than a trusted one — and decide where the double-build comparison runs so an unintended input change is caught the day it happens.

## Frame the problem "Same commit, different bytes" means some input to the build was not the same. The source was; something else was not. The work is enumerating the inputs and finding the one that varied, and Go makes that unusually tractable because the artefact records much of its own provenance. ## Step 1: read what the binaries say about themselves ``` $ shasum -a 256 app-ci app-laptop $ go version -m app-ci > ci.txt $ go version -m app-laptop > laptop.txt $ diff ci.txt laptop.txt ``` `go version -m` prints the Go version that built the binary, the module it came from, the dependency modules and their versions, and the build settings in effect — things like `-trimpath=true`, `CGO_ENABLED`, `GOARCH` and `GOOS`. In most real cases the diff of those two outputs is the whole answer: one side is a different toolchain patch release, or one side had cgo enabled, or one side was trimmed and the other was not. ## Step 2: the inputs that must match - **Toolchain version.** Different Go releases generate different code; even a patch release can. This is recorded in the binary, and it is the single most common difference between a developer laptop and a CI image. Pin it deliberately rather than letting each machine use whatever it has installed. - **Target settings.** `GOOS`, `GOARCH` and their companions. Cross-compiling to the same target from two different hosts is fine; compiling to two different targets is not the same build. - **`CGO_ENABLED` and the C toolchain.** With cgo on, a whole second compiler participates, with its own version, its own flags and its own habit of recording paths. `CGO_ENABLED=0` removes that entire class of variation, which is why reproducible-build recipes reach for it when the program allows. - **Resolved dependency versions.** The same commit should resolve identically, but only if the build actually uses the recorded requirements rather than being allowed to update them. The dependency list printed by `go version -m` makes any drift visible. - **Build tags and compiler or linker flags**, including anything supplied invisibly through `GOFLAGS`. Check `go env GOFLAGS` on both machines; a persisted `go env -w` setting on one of them is a classic silent difference. - **The build directory.** Absolute source paths are recorded in the output by default, so two checkouts in different directories differ. `-trimpath` replaces those paths with module- and import-path-based names and removes the variable entirely. Turn it on before you go looking for anything more exotic. ## Step 3: rule out the build cache, explicitly Engineers reach for "maybe the cache served something stale" early, and it is worth killing that hypothesis at once rather than leaving it as background noise. Cache entries are keyed by a hash of every input to the action; a stored result is reused only when the key matched, and if the key matched, recompiling would produce the same bytes. So the cache cannot be the source of a difference. The demonstration takes two commands — rebuild with `-a`, which reuses nothing, and hash the result; it matches the cached build. If you want the reverse experiment, `go clean -cache` and rebuild: still the same bytes. The cache is a speed mechanism with no influence on output. ## Step 4: look at what the build reads that you did not think of When the settings match and bytes still differ, the remaining suspects are inputs the source tree brings along: - **committed generated code that is not deterministic** — a generator whose output depends on map iteration order or on the time it ran will differ between regenerations, and the difference reaches the binary, - **embedded files that are not identical** on the two machines, including anything a build script writes before compiling, - **line endings or file mode differences** from a checkout configured differently on one side. At that point `go build -x` on both machines, with the outputs diffed, shows the exact command sequence each build executed, which narrows the failure to a single step rather than a whole pipeline. ## Step 5: make it stick Having achieved identical bytes once, encode it: pin the toolchain, fix `CGO_ENABLED` and the target explicitly in the release script, pass `-trimpath` there rather than trusting the environment, and have the pipeline hash the artefact and compare it against a second, independent build of the same commit. That comparison is a cheap continuous check — it fails the day someone changes a build input without meaning to, which is exactly the day you want to hear about it rather than during an incident six weeks later. The payoff is not aesthetic. Byte-identical builds mean an artefact can be re-derived from a commit and verified rather than trusted, that a cache or mirror serving you a binary is checkable, and that "the binary in production does not match what we think we shipped" becomes a question with an answer.

  • Why can you rule out the build cache as the cause of differing bytes?
    Because an entry is reused only when the hash of every input matched, and if every input matched, recompiling would produce the same output. You can show it: build normally, build again with `-a` so nothing is reused, and hash both artefacts. They agree.
  • Which single flag most often resolves this, and why is it so easy to miss?
    `-trimpath`. The compiler records absolute source paths, so `/home/alice/app` and `/builds/7/app` bake different strings into the binary. It is easy to miss because the build directory is not something anyone thinks of as an input, and nothing in the source or the command line mentions it.
  • Once builds are byte-identical, how do you keep them that way?
    Pin the toolchain, set the target and `CGO_ENABLED` explicitly in the release script, pass `-trimpath` there rather than relying on the environment, and have the pipeline build the same commit twice independently and compare hashes. That check fails the day an input changes unintentionally.
  • What makes byte-identical builds worth the effort at all?
    An artefact can then be re-derived from a commit and verified rather than trusted. It turns "is the binary in production the one we think we shipped" into a checkable question, and it makes any mirror, cache or third party that hands you a binary auditable.

saying these in an interview costs you the question

  • Blames the build cache for serving stale or different output
  • Never inspects the build settings recorded in the binary
  • Forgets that absolute source paths are compiled into the output
  • Assumes any Go version builds the same commit identically
  • Ignores CGO_ENABLED differences between a laptop and a CI image