In GitLab, how does the dependency proxy stop CI jobs from hitting Docker Hub pull rate limits, and what changes in each job's image reference?
answer
- a cache that lives on a group
- manifest HEAD checks are free
- a predefined image prefix
- top-level group or direct group
basics
~10 sThe group-level dependency proxy caches Docker Hub manifests and blobs and checks freshness with uncounted HEAD requests. Jobs prefix images with CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX (or the DIRECT variant), and the runner signs in automatically.
solid answer
~40 sThe **dependency proxy for container images** is a pull-through cache that GitLab runs per **group**, not per project, for images on Docker Hub. Docker Hub counts manifest GET requests; the proxy starts every request with a HEAD for the manifest, which Docker Hub does not count, and fetches a new image only when the cached manifest is stale, serving the blobs from GitLab. In a job you change `image: node:22-alpine` to `image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/node:22-alpine` (top-level group) or use `CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX` (the project's own group or subgroup). Runners sign in automatically; a manual login uses `CI_DEPENDENCY_PROXY_USER`, `CI_DEPENDENCY_PROXY_PASSWORD` and `CI_DEPENDENCY_PROXY_SERVER`. Since GitLab 18.0, personal and group access tokens must carry `read_registry` and `write_registry` (or the virtual-registry scopes) to use it.
code
yaml · 5 linesbuild:
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/node:22-alpine
script:
- npm ci
- npm run buildgo deeper
Recall that the dependency proxy is a group-level cache for Docker Hub images and that jobs use it by prefixing the image with CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX.
Explain the HEAD-before-GET mechanism, the difference between the top-level and direct group prefixes, automatic runner sign-in, and the Guest role needed to pull.
Roll it out across a hierarchy: enable it on the right group, set Docker Hub credentials and a TTL policy, update tokens for the 18.0 scope rules, and diagnose not-found pulls caused by missing group access.
Weigh one shared top-level cache, which maximises hits, against per-subgroup caches that keep settings and credentials separate, and decide which upstream images teams may consume at all.
## The problem it solves Docker Hub limits how many image pulls a client may make in a time window. In CI every job that names `image: node:22-alpine` pulls again, and a busy group behind one address can exhaust the limit quickly. GitLab's **dependency proxy for container images** is a local, **pull-through cache**: jobs ask GitLab for the image, GitLab fetches it from Docker Hub once, stores it, and serves later pulls from its own storage. ## How the cache decides when to call Docker Hub A container pull is two kinds of request: 1. the **manifest**, which lists the image's layers; 2. the **blobs** (layers), fetched one by one using the manifest. Docker Hub's limit counts **GET requests for manifests**. It does not count **HEAD** requests, which return the manifest's digest without its body. The proxy uses that: - every request starts with a HEAD to Docker Hub to learn the current digest; - if the cached manifest is still current, GitLab serves the manifest and blobs from its cache; - only when the manifest is stale does it pull the new image. The documentation's example: a pipeline that pulls `node:latest` every five minutes would make 360 requests in six hours directly, but through the proxy makes one pull unless the tag changed. ## Where it lives and how a job uses it The proxy is a **group** feature. It is not available for projects, it is enabled per group under the group's package and registry settings (the required role changed from Maintainer to Owner in GitLab 17.0), and an administrator can turn it off for the whole instance. Only images on Docker Hub are supported. Predefined variables make the job side short: | Variable | Meaning | |---|---| | `CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX` | image prefix through the **top-level** group | | `CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX` | image prefix through the project's **direct** group or subgroup | | `CI_DEPENDENCY_PROXY_SERVER` | the server to log in to, host and port | | `CI_DEPENDENCY_PROXY_USER` / `CI_DEPENDENCY_PROXY_PASSWORD` | credentials for a manual login | The prefixes include the server port. A hand-written reference therefore looks like `gitlab.example.com:443/acme/dependency_proxy/containers/alpine:latest`, and the port must be present unless you logged in without it. Authentication: - **Runners sign in automatically** when a job's `image` uses the proxy prefix. - Inside a job that runs Docker commands itself, such as a `FROM` line in a build, log in explicitly with the three `CI_DEPENDENCY_PROXY_*` login variables. - Outside CI, use a personal access token, group access token or group deploy token. Users need at least the **Guest** role on the group they pull through; without it pulls fail with `not found` or 404 errors that do not mention permissions. ## What changed in recent releases - **GitLab 17.10** added the option to give the proxy **Docker Hub credentials** (with UI support in 17.11), so GitLab's requests to Docker Hub are authenticated instead of anonymous, which is the default. - **GitLab 18.0** made scope checking a breaking change: tokens used against the proxy must carry `read_registry` **and** `write_registry`, or the new `read_virtual_registry` and `write_virtual_registry` scopes, which became generally available in the same release. Tokens that used to work without those scopes are rejected. - With **SAML SSO enforcement**, users must have signed in through SSO before pulling through the proxy. ## Rolling it out across a group 1. As a group Owner, turn on **Enable Proxy** on the group that will serve the cache, usually the top-level group. 2. Add **Docker Hub authentication** credentials so GitLab does not pull anonymously. 3. Change every Docker Hub reference in job `image` and `services` entries to the proxy prefix. 4. For `FROM` lines in images built inside jobs, log in to `CI_DEPENDENCY_PROXY_SERVER` first and use the full proxy path. 5. Replace any personal or group access tokens that lack the scopes GitLab 18.0 started to enforce. ## Storage and cleanup Cached blobs are kept **indefinitely** unless you remove them. Options: - turn on the group's automatic **time-to-live** policy, which clears items not pulled for 90 days by default; - clear the whole cache through the dependency proxy API, accepting that the next pipeline pulls from Docker Hub again. On a self-managed instance, standard backups do not include the proxy's files, so a restored instance can report missing files until the cache is cleared or copied. ## When it is the wrong tool - Images from registries other than Docker Hub are not proxied. - It caches pulls; it is not a place to publish your own images, which belong in the project's container registry. - Pinning by digest still works through the proxy (`.../alpine@sha256:...`) and avoids surprises when a moving tag changes upstream.
- In GitLab, a job pulling through the dependency proxy fails with `not found`, although the image exists on Docker Hub. What do you check first?Whether the user running the job has at least the Guest role in the group named in the prefix; missing group access shows up as `not found` or a 404 rather than an access error. Then check that the group has the proxy enabled and that any manually used token carries the required scopes.
- In GitLab, when would you choose `CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX` over the top-level prefix?When the proxy, its Docker Hub credentials or its access should be managed on the subgroup the project sits in rather than on the top-level group. The top-level prefix gives one shared cache for the whole hierarchy, which maximises hits but puts every project behind one group's settings.
A branch library that keeps copies of popular books and, each time someone asks, phones the central library only to ask whether a newer edition exists; it fetches a new copy only when the answer is yes.
- The dependency proxy is enabled per project, next to the container registry.
- Once cached, the proxy never checks Docker Hub for a newer image.
- Every pull through the proxy still counts as a Docker Hub pull.
- The proxy can cache images from any registry, not only Docker Hub.
- Any personal access token works with the proxy, whatever its scopes.
Assumes GitLab 19.4. Since GitLab 18.0 the dependency proxy rejects personal and group access tokens that lack read_registry plus write_registry (or read_virtual_registry plus write_virtual_registry); Docker Hub credentials for the proxy arrived in 17.10.