skip to content

Runners & Executors

2 roadmaps6 questions

GitLab's runner agent and the executor that decides how a job's environment is created — shell, Docker, or a pod per job. Interviewers ask because executor choice drives isolation, caching behaviour and whether Docker-in-Docker is even needed.

part ofGitLab CI/CDoverview, primer and where to startread it →
on this pageshow

explore

questions

6

In GitLab CI, how does a job's `tags:` keyword decide which runner picks the job up, and why can a job sit pending forever?

level: juniorimportance: must knowfreq 70%

answer

  1. labels on the runner, requirements on the job
  2. superset match, not overlap
  3. untagged jobs need an opt-in
  4. pending, not failed, when nothing matches

basics

~20 s

A GitLab CI job runs only on a runner that carries every tag listed in its tags: keyword. If no online, permitted runner has all of them, the job stays pending until such a runner appears or GitLab drops it as stuck.

solid answer

~50 s

Tags are free-form labels on the runner; `tags:` in `.gitlab-ci.yml` is the job's requirement list. GitLab schedules a job onto a runner whose tag set **contains all** the job's tags — it is an AND, not an OR, and the strings must match exactly. A job with no `tags:` at all can only land on runners that have "Run untagged jobs" enabled (`run_untagged`). When nothing matches, the job does not fail fast: it stays `pending` and GitLab shows the "stuck because you don't have any active runners online or available with any of these tags" message, then eventually drops it. The usual causes are a typo, the only tagged runner being offline or paused, the job running on a protected ref while the runner is not protected, or shared runners being disabled for the project.

code

yaml · 12 lines
yaml
build-arm:
  stage: build
  tags:
    - docker
    - arm64
  script:
    - ./build.sh

lint:
  stage: test
  script:
    - ./lint.sh

go deeper

for a junior

Know that tags on a job are requirements and tags on a runner are capabilities, and that the runner must have all of the job's tags. Be able to say why a job shows as pending.

for a middle

Explain the superset matching rule, the untagged-job opt-in, and the practical checklist for a stuck job: typo, offline runner, protected ref, runner not attached to the project.

for a senior

Show judgment about fleet design — capability tags over host names, short requirement lists, and separate runner pools for privileged or deployment work rather than one over-tagged machine everyone queues behind.

for a principal

Be ready to argue that tags are routing and not authorization, and to describe the controls that actually enforce which workloads may reach production runners across many teams and projects.

## Two halves of the same match GitLab's job routing is a set-containment test between two things that are configured in completely different places. On the **runner** side, tags are free-form labels attached when the runner is created (in the GitLab UI or API) or supplied at registration time. They describe *capabilities*: `docker`, `windows`, `arm64`, `gpu`, `deploy-prod`. On the **job** side, `tags:` in `.gitlab-ci.yml` is a list of requirements: ```yaml integration-test: stage: test tags: - docker - arm64 script: - ./run-integration-tests.sh ``` GitLab will hand that job to a runner only if the runner's tag set is a **superset** of the job's list. A runner tagged `docker, arm64, gpu` qualifies. A runner tagged only `docker` does not. There is no partial credit and no OR — adding a tag to a job makes it *harder* to schedule, never easier. ## The untagged case A job with no `tags:` key is an *untagged* job. It can only be picked up by runners whose "Run untagged jobs" option is on (the `run_untagged` field on the runner). This trips people up in both directions: - A brand-new runner created **with** tags and **without** `run_untagged` will silently ignore every job in a pipeline that never declares `tags:`. - A runner created with no tags and `run_untagged` on becomes a magnet for every untagged job in every project it is visible to. ## Why the job hangs instead of failing Unmatched jobs are not a pipeline error — GitLab assumes a suitable runner might come online in a minute, so the job sits in `pending`. Common reasons a match never happens: - **Typo or drift.** The runner's tag was renamed and the YAML still asks for the old string; matching is exact. - **The runner is offline or paused.** Only online, non-paused runners are considered. - **Protected refs.** A runner set to `ref_protected` accepts jobs only from protected branches and tags; a runner that is *not* protected is not offered jobs from protected refs when the project requires protected runners. - **Availability, not tags.** The tag matches but the runner is not attached to this project at all — shared runners disabled for the project, a project runner locked to a different project, or a group runner in a group this project isn't under. - **Concurrency.** The runner matches but is already at its configured job limit, so the job waits rather than being rejected. GitLab's UI is explicit here: the job page lists the tags the job asked for and the runners available to the project, which is the fastest way to see the mismatch. A background cleanup eventually fails long-pending jobs as stuck rather than leaving them forever. ## Tags do not choose the executor A frequent misconception is that `tags: [docker]` asks for a Docker environment. It does not. Tags select a **runner**; that runner's own `config.toml` decides whether the job runs as a shell command on the host, inside a container, or in a Kubernetes pod. The convention of tagging a Docker-executor runner `docker` is just a convention — nothing enforces that the label describes reality, which is why a job "running in Docker" can turn out to be executing shell commands directly on a build host. ## Designing a tag scheme Good tags name capabilities and trust levels, not machines: `linux`, `arm64`, `windows`, `gpu`, `prod-deploy`. Bad tags name individual hosts (`build-vm-07`), because the moment that host dies every job referencing it hangs. Keep the requirement lists short. Every extra tag on a job narrows the pool that can serve it, and a four-tag job needs a single runner carrying all four — which usually means exactly one machine, and therefore a single point of failure and a queue behind it. If you find yourself needing many tags, that is usually a sign the distinction belongs in the runner's executor configuration or in a separate runner fleet instead.

  • A job with no tags: never runs even though several runners are online. What would you check?
    Whether any of those runners has "Run untagged jobs" enabled. A runner created with tags and without `run_untagged` only accepts jobs that name its tags, so an untagged job has no eligible runner and waits. Fix it by enabling the option on one runner, or by adding an explicit `tags:` list to the job.
  • Why is tagging a runner after the machine it runs on a bad idea?
    Because the tag becomes an address rather than a capability. Jobs referencing `build-vm-07` are unschedulable the moment that host is replaced, and you cannot scale the pool horizontally without editing every pipeline. Capability tags like `arm64` or `docker` let you add and retire machines freely, since any host with the capability can carry the label.
  • Can tags alone stop an untrusted job from reaching a deployment runner?
    No. Tags are a routing hint, not an authorization boundary — anyone who can edit `.gitlab-ci.yml` can request any tag. Restrict deployment runners with project-level assignment, the protected (`ref_protected`) setting so they only take jobs from protected refs, and protected environments with approvals. Tags decide where a job *can* go; permissions decide where it is *allowed* to go.

saying these in an interview costs you the question

  • Thinking a job runs if it matches any one of its tags
  • Believing tags: [docker] makes the job run inside a container
  • Assuming an unmatched job fails immediately with an error
  • Treating tags as a security boundary for deployment runners
  • Not knowing untagged jobs need run_untagged on the runner

context

resolving links…

open as a page

In GitLab Runner's config.toml, what does the `executor` setting control, and how do the shell, docker, and kubernetes executors differ?

level: middleimportance: must knowfreq 72%

basics

~20 s

The executor decides how GitLab Runner creates the environment a job's script runs in. Shell runs commands directly on the runner host, docker starts a fresh container per job from the job's image, and kubernetes schedules a pod per job in a cluster.

open as a page

In GitLab, how do shared, group, and project runners differ, and what does that choice mean for trust and queueing?

level: middleimportance: should knowfreq 50%

basics

~20 s

Shared runners serve every project on the instance, group runners serve one group and its subgroups, and project runners are attached to specific projects. Narrowing the scope narrows who can send code to that machine, at the cost of queueing and idle capacity.

open as a page

In GitLab Runner's config.toml, what is the difference between the global `concurrent` setting and a runner entry's `limit`?

level: seniorimportance: should knowfreq 38%

basics

~20 s

concurrent is a process-wide ceiling on jobs running across every runner entry in that config.toml, while limit caps how many jobs one specific [[runners]] entry may run at once. The effective parallelism is the smaller of the two.

open as a page

A GitLab CI job on a docker-executor runner needs to build a container image. Why does the docker:dind service require privileged mode, and what are the alternatives?

level: seniorimportance: should knowfreq 55%

basics

~20 s

Docker-in-Docker starts a second Docker daemon inside the job's container, and a daemon needs kernel capabilities that ordinary containers lack, so the runner must be configured with privileged mode. Daemonless builders such as Buildah or rootless BuildKit avoid that by building images without a daemon.

open as a page

GitLab CI jobs on an autoscaled runner fleet almost never get a cache hit, even though `cache:` is configured correctly. Why, and what fixes it?

level: seniorimportance: nice to knowfreq 32%

basics

~20 s

By default GitLab Runner stores caches locally on the machine that ran the job, so a fleet that creates a fresh instance or pod per job never finds the previous cache. Configuring distributed cache storage in [runners.cache] with Shared enabled puts caches in object storage instead.

open as a page