skip to content

Dependency Proxying & Caching

A proxy repository caches public registries and decides where each request resolves. Interviewers ask about upstream outages, stale caches and how resolution order lets a rogue package in.

on this pageshow

questions

6

What can a proxy repository still serve your builds when the public registry it caches goes down on release day?

level: juniorimportance: must knowfreq 68%

answer

  1. filled on demand, not mirrored
  2. what was fetched before survives
  3. stale metadata beats no metadata
  4. assumed offline after repeated errors
  5. never-requested versions fail

basics

~20 s

A proxy repository can still serve whatever it already cached: artifact files and, in products such as Artifactory and Harbor, its last copy of the metadata. Anything no build requested through it was never stored, so it fails until the upstream returns.

solid answer

~50 s

A proxy repository caches **on demand**: it stores only what some build already requested through it. When the upstream goes down, cached artifact files keep being served, and so does the last cached copy of metadata such as `maven-metadata.xml` or an npm packument, now stale. Anything never requested through it fails, including a version published upstream yesterday. Artifactory documents the mechanics: after a connection error it retries up to 10 times, then treats the remote repository as *assumed offline* for the Assumed Offline Period (default 300 seconds) and serves cached copies; a metadata refresh that exceeds the Metadata Retrieval Cache Timeout returns the previous metadata. Harbor's proxy cache project likewise serves the cached image when the target registry is unreachable. So outage resilience is decided beforehand: every build goes through the proxy, and cleanup must not evict what releases need.

go deeper

for a junior

Recall that a proxy repository caches only what was requested through it, that cached content keeps being served when the upstream is down, and that never-requested versions fail.

for a middle

Explain the two kinds of cached content, artifact files and metadata, why stale metadata is served during an outage instead of nothing, and how the assumed-offline window and its retries work.

for a senior

Show you prepare before the outage: every build through the proxy, cleanup that does not evict release-critical dependencies, hosted copies of what cannot be lost, and the Offline switch when an upstream hangs.

for a principal

Weigh how much outage tolerance the organisation needs against storage and operating cost: long cache retention, hosted copies of critical dependencies and a fully disconnected repository each buy more independence for more work.

## What a proxy repository actually holds A **proxy repository** sits between your builds and a public registry such as Maven Central, the npm registry, PyPI or Docker Hub. Each product has its own word for it: Artifactory calls it a *remote* repository, Nexus Repository a *proxy* repository, and Harbor a *proxy cache project*. In every case it is a cache filled **on demand**, not a copy of the registry. Artifactory's documentation says so directly: a remote repository acts as a proxy, not as a mirror, and artifacts are fetched and cached only when a client requests them. That one fact decides almost everything about an outage. The proxy holds two kinds of content: - **Artifact files**: the JAR, the npm tarball, the wheel, the image layer. A released version's file is expected never to change, so a cached copy stays valid. - **Metadata**: the documents that list what exists, such as `maven-metadata.xml`, an npm packument with its `dist-tags`, a PyPI simple index page, or the manifest a container tag points to. These change whenever the upstream publishes, so the proxy refreshes them on a timer. ## What happens when the upstream stops answering Artifactory documents the sequence, and it is a good model of the idea: 1. A request arrives for something not cached, or for metadata whose cache period has run out, so the remote repository calls the upstream. 2. The call fails. Artifactory retries, with the interval growing each time, up to 10 times. 3. If every retry fails, the repository is treated as **assumed offline** for the *Assumed Offline Period* (default 300 seconds). During that window it stops calling the dead upstream and serves the stale copies it already holds. 4. When a metadata refresh does not finish within the *Metadata Retrieval Cache Timeout* (default 60 seconds), the previous metadata is returned to the client instead. 5. After the offline period an online check runs; if the upstream is back, normal caching resumes. Harbor's proxy cache project describes the same choice for container images: if the target registry is not reachable, the proxy cache project serves the cached image. ## What still works and what fails | Request during the outage | Outcome | |---|---| | A version an earlier build downloaded through the proxy | Served from the cache | | Metadata the proxy cached earlier | Served stale, without versions published since | | A version published upstream but never requested through the proxy | Fails, because it was never stored | | A package no build has ever asked for | Fails | | A cached version that a cleanup run removed | Fails until the upstream returns | The stale-metadata row deserves attention. A build that resolves "the newest matching version" chooses from the cached version list, so during the outage it may quietly build against an older version than it would have the day before. On release day that is usually what you want; it is still worth knowing it happened. ## Preparing before the outage, not during it - **Route every build through the proxy**, release builds included. A build that talks to the public registry directly gets no protection, and it never warms the cache for anyone else. - **Watch what cleanup evicts.** Artifactory's *Unused Artifacts Cleanup Period* removes cached artifacts nobody has used for a while; leaving the field empty keeps them indefinitely. Evicting a release-critical dependency the week before an outage turns a cache hit into a failure. - **Use the Offline switch deliberately.** Artifactory's per-repository *Offline* checkbox makes a remote repository use only what is already cached, with no further fetch attempts. That helps when the upstream hangs rather than refuses, so builds stop waiting on timeouts. - **Copy what you cannot lose.** For the few dependencies a release cannot ship without, a copy deployed into a hosted repository no longer depends on the remote cache surviving. ## Common misconceptions - "The proxy mirrors the registry." It stores only what was requested through it. - "An outage switches the proxy off." Cached content keeps flowing; only uncached requests fail. - "Clearing the cache will help." In Artifactory, a zap only marks metadata as expired, and when the refresh cannot reach the upstream the previous copy is served, so it gains nothing; actually deleting cached files during an outage throws away the only copy you have.

  • What does Artifactory's Offline checkbox on a remote repository change during an outage?
    It stops that remote repository contacting its upstream: only artifacts already in its cache are used, and no further fetch attempts are made. That turns slow timeouts into fast answers when the upstream hangs instead of refusing connections, at the price of seeing nothing new until you clear the checkbox. Global Offline Mode does the same for every remote repository at once.
  • How do you make sure the cache already holds what a release needs before the upstream fails?
    Send every build, CI and release alike, through the proxy so dependencies are cached as a side effect of normal work. Keep the unused-artifact cleanup period long, or empty for indefinite retention, on the remote repositories releases depend on. Deploy copies of the few dependencies a release cannot ship without into a hosted repository, which no upstream outage or remote-cache cleanup removes.

A proxy repository is like a corner shop that restocks only items customers have asked for before: when the wholesaler closes, the shelves still hold earlier orders, but nothing nobody ever bought.

  • A proxy repository mirrors the whole public registry, so an outage changes nothing.
  • Once the upstream is down, the proxy stops serving even what it has cached.
  • Clearing the proxy's cache during an outage forces a fresh, current download.
  • Deleting cached artifacts is harmless because the proxy can always fetch them again.
open as a page

Why does a proxy repository recheck cached metadata, such as a Maven version list, much sooner than the artifact files it caches?

level: middleimportance: must knowfreq 58%

basics

~20 s

A released artifact file is expected never to change, so its cached copy stays valid; metadata such as maven-metadata.xml or an npm packument changes every time the upstream publishes, so the proxy must refresh it to see new versions.

open as a page

Artifactory's npm virtual repository lists the local repository first, yet a build got a public package carrying an internal name; why didn't order protect it?

level: seniorimportance: must knowfreq 55%

basics

~20 s

Order only says where a given file is looked for first; version metadata is merged from every member, so a higher public version of the same name can win. Exclude patterns or Priority Resolution, not order, keep internal names off the remote.

open as a page

Why can a proxy repository keep returning 404 for a package version that was published upstream only minutes ago?

level: middleimportance: should knowfreq 42%

basics

~20 s

A proxy repository remembers not-found answers for a while: a request made before the version existed cached the 404, and until that period ends the proxy does not ask the upstream again, so the new version stays invisible.

open as a page

How does an OCI registry running as a pull-through cache decide whether its cached copy of an image tag is still current?

level: middleimportance: should knowfreq 46%

basics

~20 s

On a pull by tag, a pull-through cache asks the upstream which manifest the tag points to now and fetches the new content if the tag moved; a pull by digest names fixed content, so the cached copy needs no freshness check.

open as a page

Should your organisation air-gap dependency resolution behind an internal repository manager, and what does cutting the public upstream commit you to?

level: principalimportance: should knowfreq 33%

basics

~20 s

Air-gapping resolution removes the live upstream, so nothing new arrives except through a transfer you run; it buys isolation at the cost of an import pipeline, delayed versions and an emergency-patch path that someone must own.

open as a page