skip to content

Before cutting a release built offline from vendor/, how do you prove the vendored tree matches go.mod?

level: seniorimportance: should knowfreq 34%

answer

  1. one failure is loud, one is silent
  2. the build only checks the index, not the files
  3. regenerate somewhere the sources are reachable
  4. version control is the diff tool here
  5. run it per pull request, not at release

basics

~20 s

Re-run go mod vendor where the module sources are reachable, then confirm version control reports no change under vendor. That regeneration is the only proof the tree matches go.mod: the build's own check compares modules.txt with go.mod and never reads the copied files.

solid answer

~40 s

Two things can be wrong, and only one fails the build. The loud one is `vendor/modules.txt` disagreeing with `go.mod`; the `go` command catches that itself and refuses to start. The silent one is the copied source not being what those versions contain — a hand-applied patch under `vendor/`, a bad merge, a partial commit that took the index but not the files. Nothing at build time detects that, because vendored source is compiled as-is and is never re-hashed. So the release check has to be a regeneration check: on a machine that can reach the module sources, re-run `go mod vendor` and require `git status --porcelain vendor` to come back empty. Run it on every pull request, not only at release, so drift is caught while its diff is still small.

code

text · 5 lines
text
$ go mod vendor
$ git status --porcelain vendor
 M vendor/modules.txt
?? vendor/golang.org/x/text/
# any output at all means the committed tree had drifted

go deeper

for a junior

The takeaway to hold on to: a build succeeding is not evidence that the vendored directory is correct. Regenerating it and seeing no changes is.

for a middle

Be ready to explain the two distinct failure modes — an index that disagrees with go.mod, which the build catches, and copied files that are not what the versions contain, which it does not.

for a senior

An interviewer expects the concrete gate: regenerate on a machine that can reach the module sources, fail on any version-control diff, run it per pull request, and pin the toolchain so the check does not report phantom drift.

for a principal

Decide what the check is worth and who maintains it. Committing a vendor directory without a regeneration gate buys an offline build whose contents nobody has ever verified — the guarantee you claimed is not the one you have.

## The scenario The release build runs on a machine with no network access. That is the whole reason the repository carries `vendor/`: everything the compiler needs is in the commit, so the artifact can be produced from a checkout and nothing else. The question a release engineer has to answer before pressing the button is not "does it build" — it will — but **"is what is in `vendor/` actually the dependency set `go.mod` declares?"** ## Two failure modes, only one of which the toolchain catches **The loud one.** `vendor/modules.txt` and `go.mod` disagree: a module required but not marked explicit, a module marked explicit but no longer required, or a version mismatch. The `go` command performs this comparison before compiling and stops with `go: inconsistent vendoring`, listing every offender. You cannot miss it, and you cannot ship past it. **The silent one.** The index and `go.mod` agree perfectly, but the *files* are not what those versions contain. This happens more often than people expect: - someone patched a dependency bug by editing a file under `vendor/` and committed it; - a merge conflict in the vendored tree was resolved by hand, badly; - a commit included `vendor/modules.txt` and `go.mod` but only some of the moved source files; - a regeneration was run on a machine that could not reach some modules, producing an incomplete tree that still indexes consistently. None of these fail the build. The `go` command compares the index against `go.mod` and stops there; it does not re-hash the vendored source, because in vendor mode that source *is* the input, not a cache of something verifiable. A one-line edit compiles exactly like the original and leaves no trace outside version-control history. ## The check that closes the gap Because the only authority on what those versions contain lives outside the repository, verification requires a machine that can reach the module sources. There the check is simple and mechanical: ``` go mod vendor git status --porcelain vendor ``` `go mod vendor` resets the directory from `go.mod`, so if the committed tree was faithful, the regeneration produces byte-identical files and version control reports nothing. Any output at all — a modified file, an untracked file, a deletion — is drift, and the output itself names exactly which package drifted. This is the diagnostic, and it doubles as the fix: the same command that detects the problem has already corrected it on disk. You commit the result. ## Where to run it Run it on **every pull request**, not at release time. The regeneration check is cheap, and its value collapses if you defer it. Caught on the PR that introduced it, drift is a two-line diff with an obvious author and an obvious cause. Caught at release, it is an unexplained difference somewhere in a tree of tens of thousands of lines, discovered by the person least able to say whether the edit was deliberate. A release-time run is still worth having as a backstop, because branches merge and CI configurations change. ## Making the check trustworthy Two things make this check misbehave, and both are worth pre-empting. **Toolchain skew.** The tree is generated by a specific `go` version, and different versions can write slightly different output — annotations in `modules.txt` in particular. If CI regenerates with a different toolchain than the developers use, the check reports drift on every commit and gets disabled within a week. Pin the toolchain the check runs with. **A silent partial fetch.** If regeneration cannot reach some module and the command fails, do not let the pipeline continue to the `git status` step and conclude that nothing changed, or worse, that a truncated tree is correct. Fail the job on the regeneration command's own exit status first. ## What this does and does not prove It proves the committed tree is what `go mod vendor` produces from the current `go.mod` **on the machine that ran the check, at that moment**. That is a strong property and the right one for a release gate. It is not an independent audit of the dependencies themselves — it says the tree matches the declared versions, not that those versions are safe, and not that the vendored copies came from a source you trust more than the one CI reached. Vendoring moves the trust decision to the moment of regeneration; the check just makes sure nothing has moved since. ## The payoff With the check in place, the offline property is real: any commit on the release branch can be rebuilt on an isolated machine, months later, with no dependency on any remote host still serving those versions. That guarantee is the entire reason to carry `vendor/`, and without a regeneration check it is a guarantee about a directory nobody has verified.

  • Why doesn't the go command detect a hand-edited file under `vendor/`?
    Because it never re-hashes the vendored tree. The consistency check compares `vendor/modules.txt` with `go.mod` — module paths, versions, explicit markers — and stops there. In vendor mode the copied source is the build's input rather than a cache of something verifiable, so a one-line edit compiles exactly like the original.
  • What makes the regeneration check flaky in CI, and how do you stop that?
    It compares against whatever the toolchain running it produces, so a runner on a different Go version can emit a slightly different tree and report drift on every commit. Pin the toolchain the check uses. Also fail on the regeneration command's own exit status, so an unreachable module cannot leave a truncated tree that then compares clean.
  • What is the operational payoff of insisting on the offline build?
    The artifact depends on nothing outside the commit. A release engineer can rebuild last quarter's release commit on an isolated machine months later and get exactly the same dependency source, without depending on any remote host still serving those versions. That property is the main reason a team carries a vendor directory at all.
  • A dependency has a bug you need fixed today. Is editing vendor/ ever the right move?
    As a temporary local diagnosis, yes; as a committed fix, no. The next `go mod vendor` deletes it silently and nothing warns anyone, so the fix has a random expiry date. The durable options are to move to a version that contains the fix, or to carry the change as a fork the module graph points at deliberately.

saying these in an interview costs you the question

  • Says a vendored build verifies files against recorded hashes
  • Assumes a green build proves the vendored tree is faithful
  • Plans to run go mod vendor on the offline release machine
  • Checks only that the vendor directory exists
  • Commits a patch to a vendored file as the permanent fix
  • Runs the regeneration check only at release time