In a CircleCI workflow, a downstream job cannot find the build output an earlier job produced. How do CircleCI's workspaces, caches, and artifacts differ, and which one should carry that output?
answer
- each job starts on a clean executor
- three mechanisms, three different scopes
- one is per-workflow-run, one spans runs
- a cache key is never overwritten
- persist_to_workspace and attach_workspace
basics
~20 sEvery CircleCI job starts on a clean executor, so nothing carries over implicitly. Workspaces move files between jobs in one workflow run and are the right tool here; caches speed up refetching dependencies; artifacts are outputs stored for people to download.
solid answer
~50 sEach job in a workflow gets a fresh executor, so files do not travel unless you move them. The **workspace** is the mechanism for that: the producing job runs `persist_to_workspace` with a `root` and `paths`, and each downstream job runs `attach_workspace` at a chosen path. It is scoped to a single workflow run and flows only along `requires` edges, which is exactly the semantics of passing build output forward. A **cache** is a different thing — `save_cache` with an explicit `key` and `restore_cache` with a list of keys — scoped across runs and branches, intended for dependencies you could refetch if it vanished. Crucially a cache key is write-once: saving to a key that already exists is a no-op, so a cache used to pass build output silently serves the first version forever. **Artifacts**, stored with `store_artifacts`, are for humans and external consumers after the run, not for feeding later jobs.
code
yaml · 41 linesversion: 2.1
jobs:
build:
docker:
- image: cimg/node:20.11
steps:
- checkout
- restore_cache:
keys:
- v1-deps-{{ checksum "package-lock.json" }}
- v1-deps-
- run: npm ci
- save_cache:
key: v1-deps-{{ checksum "package-lock.json" }}
paths:
- ~/.npm
- run: npm run build
- persist_to_workspace:
root: .
paths:
- dist
- store_artifacts:
path: coverage
destination: coverage-report
deploy:
docker:
- image: cimg/node:20.11
steps:
- attach_workspace:
at: .
- run: ./scripts/deploy.sh dist
workflows:
ship:
jobs:
- build
- deploy:
requires:
- buildgo deeper
Remember that every job starts clean, so files do not appear on their own. Know that persist_to_workspace and attach_workspace are the pair that moves files from one job to another.
Distinguish the three mechanisms by scope: workspace within one workflow run, cache across runs, artifacts for after the run. Explain restore_cache prefix matching and why cache keys include a checksum.
Diagnose out loud: check persist paths, the attach path, the requires edge, then whether a cache is doing a workspace's job. Name the write-once cache-key rule as the reason the failure is intermittent rather than immediate.
Own the discipline behind it: build the deployable once and let every later job consume identical bytes, decide what may be cached at all, and set a key-versioning convention so a suspected poisoned or stale cache can be invalidated fleet-wide in one change.
## Start from the invariant Every job in a CircleCI workflow runs on its own freshly provisioned executor. The `checkout` step gives it source code; nothing else from a previous job is present. Any file that needs to cross a job boundary must be handed across deliberately. CircleCI gives you three mechanisms for moving files, and confusing them is one of the most common real-world configuration bugs — each failure mode is distinct and each is quietly misleading. ## Workspaces: within one workflow run A workspace is the correct answer for build output. The producing job persists paths; consuming jobs attach them. ```yaml jobs: build: docker: - image: cimg/node:20.11 steps: - checkout - run: npm ci && npm run build - persist_to_workspace: root: . paths: - dist deploy: docker: - image: cimg/node:20.11 steps: - attach_workspace: at: . - run: ./scripts/deploy.sh dist workflows: ship: jobs: - build - deploy: requires: - build ``` The properties that matter: a workspace is **scoped to one workflow run**, it is **additive** (several upstream jobs can each contribute paths), and content flows only from jobs a consumer actually depends on through `requires`. `root` is the directory the listed `paths` are relative to; `at` is where the consumer materializes them. Forgetting `requires` is the classic bug — the deploy job may start before build finishes, attach an empty or partial workspace, and fail with a missing-file error that looks like a build problem. Workspaces also make **build-once, promote-many** possible inside a pipeline: compile the artifact in one job and let test, scan and deploy jobs all attach the identical bytes rather than each rebuilding. ## Caches: across runs, for things you could refetch A cache is saved and restored by explicit key: ```yaml - restore_cache: keys: - v1-deps-{{ checksum "package-lock.json" }} - v1-deps- - run: npm ci - save_cache: key: v1-deps-{{ checksum "package-lock.json" }} paths: - ~/.npm ``` `restore_cache` takes an ordered list and uses the first key that matches; a key that is a prefix of stored keys performs a partial match, which is why the second, looser entry acts as a fallback when the lockfile changed. Template helpers such as `{{ checksum "file" }}`, `{{ .Branch }}` and `{{ arch }}` build keys that change when their inputs change. The leading `v1-` is a manual escape hatch: bump it to invalidate everything at once. The property that produces incidents: **a cache key is write-once**. If `save_cache` targets a key that already exists, the save is skipped rather than overwriting. That is exactly right for content-addressed dependency caches — the content for a given lockfile checksum never needs to change — and exactly wrong for anything mutable. A cache whose key does not incorporate its inputs will keep serving the first content ever stored under it, forever, with no error anywhere. ## Artifacts: outputs for people `store_artifacts` uploads files for retention and download after the run: ```yaml - store_artifacts: path: coverage destination: coverage-report ``` Artifacts are for coverage reports, binaries you want to hand to someone, logs and screenshots from a failed test. They are not automatically injected into any later job; a subsequent job wanting them would have to fetch them deliberately. Treating `store_artifacts` as the way to pass files between jobs is the third variant of the same confusion. ## Diagnosing the reported failure Work the list in order. Does the producing job actually run `persist_to_workspace`, and do the listed `paths` exist relative to `root` at that moment (a build that writes to `build/` while the config persists `dist` fails here)? Does the consuming job run `attach_workspace`, at a path its later steps actually reference? Is there a `requires` edge from consumer to producer — without one the workflow may run them concurrently? And is a cache being used where a workspace belongs, which is the version that fails intermittently rather than always, because the answer depends on what happens to be stored under that key. ## Choosing, in one line each - Cross-job files inside one run, must be exact: **workspace**. - Re-fetchable dependencies you want faster next time, correctness unaffected if absent: **cache**. - Output a human or an external system reads after the run: **artifact**.
- What happens if save_cache targets a key that already exists?The save is skipped — CircleCI does not overwrite an existing cache key. That is correct for content-addressed dependency caches, and it is why a poorly designed key that never changes will serve the same stale content indefinitely with no error. The fix is to include the inputs in the key, plus a manual version prefix you can bump.
- Why does a deploy job sometimes attach an empty workspace even though the build job persists to it?Usually a missing `requires` edge. Workspace content flows along workflow dependencies, so a job with no dependency on the producer may start concurrently and attach nothing. The other common cause is a path mismatch: `paths` are relative to `root`, and persisting a directory the build never wrote produces an empty attach.
- Can a workspace carry files from one workflow run to the next?No — a workspace is scoped to a single workflow run. Anything that must survive across runs belongs in a cache if it is re-fetchable, in artifact storage if a human needs it, or in an external registry if it is a real release artifact that later pipelines will promote.
saying these in an interview costs you the question
- Assuming jobs in one workflow share a filesystem
- Using a cache to pass build output between jobs
- Expecting save_cache to overwrite an existing key
- Thinking stored artifacts are injected into later jobs
- Omitting requires and blaming flaky infrastructure