Why can a proxy repository keep returning 404 for a package version that was published upstream only minutes ago?
answer
- the proxy remembers misses
- a request that came too early
- two caches can hide a version
- missed retrieval period, 1,800 seconds
basics
~20 sA proxy repository remembers not-found answers for a while: a request made before the version existed cached the 404, and until that period ends the proxy does not ask the upstream again, so the new version stays invisible.
solid answer
~50 sTwo caches can hide a fresh version. The **negative cache** remembers that a path returned `404`: in Artifactory, the *Missed Retrieval Cache Period* (default 1,800 seconds) means no new upstream request is made for that path until it expires, and a value of 0 turns it off. If any build, or a downstream pipeline that started too early, asked for the version before the publish finished, that 404 is now cached. Separately, the **metadata cache** may still hold a version list without the new version. To tell them apart, request the exact file path through the proxy and directly from the upstream: a file that exists upstream but 404s through the proxy points at the negative cache. The fix is a shorter period on remote repositories where fresh versions matter, accepting more upstream traffic for paths that genuinely do not exist.
go deeper
Recall that a proxy can remember a not-found answer for a while, so a version published after an early request can stay invisible through the proxy.
Explain the negative cache, why it exists, its Artifactory setting and default, and how it differs from the metadata cache in the symptom it produces.
Diagnose a stuck version by comparing the exact path and metadata through the proxy and upstream, and fix the publish-consume race rather than only lowering the period.
Set negative-cache periods per upstream from how quickly teams need new versions against the upstream traffic that probing builds generate.
## What a negative cache is and why it exists A **negative cache** (Artifactory calls it the *missed retrieval* cache) stores the fact that a path was **not found** upstream. While the entry lives, the proxy answers `404` itself instead of asking the upstream again. Proxies need this because clients ask for missing paths all the time: - a build tool that tries each configured repository in turn, so every repository except one answers "not here"; - optional files a package never had, such as extra checksum or classifier files; - mistyped names and versions that were never published. Without a negative cache each of these misses is a round trip to the public registry, slowing builds and loading the upstream. In Artifactory, the *Missed Retrieval Cache Period* defaults to **1,800 seconds**; during it, Artifactory issues no new request for an artifact that previously returned `404`. A value of `0` means 404s are not cached and every miss is asked of the upstream. ## How a fresh version gets stuck The classic failure is a race between a publish and a consumer: 1. A library's release pipeline starts publishing version 3.2.0 to the public registry. 2. A downstream build, triggered at the same moment, asks the proxy for the 3.2.0 file before the publish is visible upstream. 3. The upstream answers `404`; the proxy caches that answer. 4. Minutes later the version is live upstream, but every request through the proxy gets the cached `404` until the missed-retrieval period ends. 5. Engineers check the public registry in a browser, see the version, and conclude the proxy is broken. The proxy is behaving as configured. The 404 was true when it was cached. ## Telling the two caches apart | Symptom through the proxy | Direct from the upstream | Likely cause | |---|---|---| | Exact file path returns `404` | File downloads | Negative cache holding an early miss | | File path works, but resolution never picks the new version | Metadata lists it | Stale metadata cache | | Both fail | Both fail | Version not actually published | The test is cheap: request the exact file path and the metadata (for example the artifact's `maven-metadata.xml` or the npm packument) through the proxy, then the same paths from the upstream. ## Fixing it - **Wait it out** if the release is not urgent; the entry expires on its own. - **Lower the period** on the remote repositories where fresh versions matter. For a refresh that fails after a zap, Artifactory's own guidance is to lower `missedRetrievalCachePeriodSecs`, together with the assumed-offline period, to reduce the blast radius. - **Do not count on a zap to clear it.** Artifactory documents zap as invalidating cached metadata; it does not say zap clears remembered 404s, so the period setting is the documented lever. - **Fix the race at the source.** Make downstream builds wait until the version is visible upstream before they resolve it, so no early request caches a 404. ## Where it meets the other proxy settings - The **metadata cache** decides when version lists refresh. It hides a new version from resolution, but not from a direct request for the file. - The **assumed-offline** state that follows repeated connection errors stops the proxy calling the upstream for its period (default 300 seconds in Artifactory), so a version first requested during that window is not fetched either. - **Cleanup of unused cached artifacts** removes files that exist; it has nothing to do with remembered misses, which is why emptying a cache rarely fixes a stuck 404. ## Choosing the period - A **long** period protects the upstream and speeds up builds that probe many repositories, but widens the window in which a newly published version is invisible. - A **short or zero** period makes new versions visible immediately, but every probe for a path that does not exist becomes an upstream request. - **Different upstreams deserve different values.** A remote repository pointing at another internal instance, where releases land many times a day, can justify a short period; a public registry usually does not. ## Common misconceptions - "A 404 from the proxy proves the version does not exist." It proves only that the proxy is not looking right now. - "Re-publishing upstream will clear it." The proxy is not asking the upstream, so it cannot notice. - "The negative cache and the metadata cache are one setting." They are separate, and they produce different symptoms.
- Why have a negative cache at all, if it can hide fresh versions?Clients routinely ask a proxy for paths that do not exist upstream: a build trying each configured repository in turn, optional checksum or classifier files, mistyped names. Without remembering the 404, every such miss becomes a round trip to the public registry, slowing builds and loading the upstream. The negative cache trades a bounded window of staleness for that traffic.
- How do you stop release pipelines from tripping over the negative cache?Make the downstream build wait until the new version is visible upstream before it resolves it, so no early request caches a 404. On remote repositories that front a fast-moving internal upstream, shorten the missed-retrieval period. For public registries the default is usually acceptable, because a version that is missing for half an hour rarely blocks anyone.
It is like a receptionist told there is no guest called Ortiz, who keeps telling every caller the same thing for half an hour, even after Ortiz has checked in.
- A 404 from the proxy proves the version does not exist upstream.
- Re-publishing the version upstream will clear the proxy's 404.
- The negative cache and the metadata cache are the same setting.
- Turning the negative cache off costs nothing.