How would you set up vcpkg binary caching for a team, deciding where the cache lives, who may write to it, and how to keep hit rates high?
answer
- trusted writers, many readers
- pull requests read only
- ABI hash inputs drive hits
- experimental providers can change
basics
~20 sGive trusted CI builds of protected branches readwrite access and everyone else, including pull-request builds and developers, read access. Choose stable storage the team can authenticate to, and pin compilers, images and baselines, because they decide the ABI hash and so the hit rate.
solid answer
~50 sThere is no single right setup, but the decisions are always the same. **Where**: a NuGet feed (for example GitHub Packages) or an `http` server fits teams with existing package infrastructure; a `files` share suits one office; cloud bucket providers are marked experimental and may change, which matters for a pipeline you expect to run for years. **Who writes**: the vcpkg docs recommend read-write for CI and read-only for developers; I would narrow writes to CI on protected branches, because a cached package is trusted by hash alone and a pull request can modify ports or triplets. **Hit rate**: the ABI hash covers compiler, CMake version, triplet and port files, so pinned runner images, a shared baseline and few triplets keep hits high; `VCPKG_DISABLE_COMPILER_TRACKING` raises hits by giving up safety and is discouraged. **Cost**: storage grows per triplet and per image change, so plan retention, and measure restored versus built counts from the logs.
go deeper
Recall that a shared binary cache lets CI and developers reuse built packages instead of rebuilding them.
Explain the providers available and the read, write and readwrite modes, and why ABI hash inputs decide whether a package is restored.
Show how you diagnose low hit rates and secure write access, separating pull-request builds from trusted branch builds.
Own the trade-off between speed, storage cost and trust: choose the backend, the writer set, the pinning policy and retention, and justify each against the team's release risk.
## The decision a lead owns Binary caching turns a 40-minute dependency build into a minute of downloads, but only if the cache is shared, trusted and hit. Each of those is a choice with trade-offs; this is a judgment call, not a configuration recipe. ## Where the cache lives | Option | Strengths | Costs | |---|---|---| | NuGet feed (for example GitHub Packages) | recommended by the vcpkg docs for GitHub-hosted CI; per-feed permissions | NuGet credential setup on every agent | | `http` provider to a team server | any storage behind a URL template; header-based auth | you run and secure the server | | `files` on a network share | simplest; no extra tooling | needs a reachable share, file-level permissions | | `x-aws`, `x-gcs`, `x-azblob`, `x-cos` | uses existing cloud buckets | marked experimental: may change or be removed without warning | The removal of the GitHub Actions cache provider is the cautionary example: pipelines built on an experimental backend lost their remote cache when it went away. For a setup expected to last, prefer a non-experimental provider or wrap the experimental one in a review date. ## Who may write The vcpkg docs recommend a read-write cache for each CI pipeline and read-only access for developers. A stricter split is usually worth it: - **CI on protected branches**: `readwrite`. These builds come from reviewed code with controlled runners. - **Pull-request builds**: `read` only. A pull request can change a port, an overlay or a triplet; letting it upload means unreviewed binaries enter the cache. - **Developers**: `read`. Their machines and compilers vary, and their uploads would mostly be noise. The reasoning: a restore trusts the archive purely because its hash matches. Anyone with write access can place an archive under any hash, so the set of writers is the trust boundary of every build that restores from the cache. ## Keeping the hit rate high Hits require identical ABI hash inputs, so hit rate is mostly a matter of **pinning**: 1. **Runner images and compilers**: the compiler identity and the CMake version are hashed; an unpinned image that updates weekly forces weekly cold builds. 2. **The baseline**: one shared baseline across repositories lets them share packages. 3. **Triplets**: every extra triplet is a separate universe of binaries; support few, and name custom ones consistently. 4. **Developer parity**: developers hit CI's archives only with the same compiler and triplet; a toolset mismatch means local builds. `VCPKG_DISABLE_COMPILER_TRACKING` removes the compiler from the hash and raises hit rates, but the docs warn it can restore ABI-incompatible binaries; treat it as a last resort. ## Related supply-chain controls - **Asset caching** (`X_VCPKG_ASSET_SOURCES`) mirrors source downloads and tools; with `x-block-origin` builds fail rather than reach the original internet locations, which keeps builds working when an upstream site disappears. - **A pinned vcpkg checkout** and baseline make the port files, and therefore the hashes, reproducible. - **Private ports** belong in a registry the team controls; that design sits with registries. ## Measuring and maintaining it - Track restored versus built counts per job; vcpkg's log reports how many packages were restored from each provider and how long it took. - Plan retention: archives accumulate per triplet, compiler and port version, so decide up front how old ones are pruned on whichever storage you chose. - Re-examine after each toolchain upgrade: expect one cold run per image change, and schedule it on the protected branch so pull requests start warm. ## Rolling it out 1. Pick the backend and create two credentials: one with write access for protected-branch CI, one read-only for everything else. 2. Configure `VCPKG_BINARY_SOURCES` centrally in the CI definitions, starting with `clear`, so no job silently falls back to a throwaway local folder. 3. Run one warm-up build per supported triplet on the protected branch to populate the cache. 4. Point developers at the read-only source through their shell profile or a documented script, and check that their compiler matches the CI image. 5. Review the hit rate after a few weeks and after every image change, and prune archives that no supported configuration can still produce. ## A defensible default For most teams on a hosted CI: a NuGet or HTTP cache, CI on the main branch writing, everything else reading, pinned runner images, one baseline shared across repositories, and a short list of supported triplets.
- Why should pull-request builds not write to the shared binary cache?A pull request can change ports, overlays or triplets before review, and vcpkg trusts a cached archive purely by its ABI hash. Letting those builds upload means unreviewed binaries can be restored by later trusted builds. Read-only access still gives pull requests the speed benefit.
- Developers rarely hit the CI cache. What is the usual cause?Different ABI hash inputs: another compiler version or toolset, a different triplet, or a different baseline. Aligning the developer toolchain with the CI image, or providing a container or preset that matches it, fixes most misses.
- Every developer should have readwrite access so the cache fills faster.
- The ABI hash proves who built an archive, so write access does not matter.
- Disabling compiler tracking is a safe way to raise the hit rate.
- Experimental cloud providers are as stable as the files and nuget providers.
- One cache per developer machine is enough for a team.