In Bitbucket Pipelines, what does each step run in, and why is a file built in one step missing in the next?
answer
- one container per step
- image set per step or globally
- fresh clone, empty workspace
- declare paths to carry forward
- dependencies versus build outputs
basics
~20 sEach Bitbucket Pipelines step starts a separate Docker container from its image with a fresh clone, so nothing on disk survives the step. Files a later step needs must be declared as artifacts; caches only restore downloaded dependencies.
solid answer
~40 sEvery `step` in `bitbucket-pipelines.yml` runs its `script` in a new Docker container, started from the step's own `image` or the top-level `image` (the default build image is `atlassian/default-image:latest`), with the repository cloned into `$BITBUCKET_CLONE_DIR`. When the step finishes, that container is discarded, so a `dist/` folder built in step one does not exist in step two. To hand files forward you list them under the step's `artifacts` as glob paths relative to the clone directory; later steps download them by default, and `download: false` lets a step skip that. Artifacts are kept for at most 14 days and limited to 1 GB each. Caches look similar but do a different job: they restore downloaded dependencies between runs and may be cleared at any time, so build output belongs in artifacts, not in a cache.
code
yaml · 19 linesimage: acme/build-env:2.1
pipelines:
default:
- step:
name: Build
script:
- ./build.sh --out dist/
artifacts:
- dist/**
- step:
name: Lint only
artifacts:
download: false
script:
- ./lint.sh
- step:
name: Test the build
script:
- ./test.sh dist/go deeper
Recall that every Bitbucket Pipelines step is a new Docker container with its own fresh clone, and that artifacts are how files reach later steps.
Explain how the image is chosen, that artifact paths are relative to the clone directory, what download: false does, and how artifacts behave around a parallel group.
Show that you keep artifacts small and deliberate, never rely on a cache for correctness, and plan manual gates around the 14-day artifact retention.
Weigh fewer, larger steps against many small ones: isolation and parallelism on one side, repeated container start, clone and download cost on the other.
## One container per step Bitbucket Pipelines is Bitbucket Cloud's built-in CI. A pipeline is a list of **steps** declared in `bitbucket-pipelines.yml` at the root of the repository, and the documentation is explicit about what a step is: *each step in your pipeline will start a separate Docker container to run the commands configured in the `script` option*. Which image that container starts from is resolved in this order: 1. The step's own `image`, if it sets one. 2. Otherwise the top-level `image` that applies to every step. 3. Otherwise Atlassian's default build image, `atlassian/default-image:latest`. Before the script runs, the repository is cloned into the directory held in `$BITBUCKET_CLONE_DIR`. The clone is shallow by default (`clone: depth` defaults to 50 commits), and it happens again for every step, because every step is a new container. ## What survives the end of a step When the script finishes, the container is thrown away. That single fact explains most "it worked in the build step, why is the file gone?" surprises. | What the step produced | Visible to the next step? | How to carry it forward | |---|---|---| | Files inside the clone directory | No | List them under `artifacts` | | Files elsewhere, such as a home-directory tool cache | No | A cache, and only for later pipeline runs | | Shell variables exported in the script | No | Write `NAME=value` to `$BITBUCKET_PIPELINES_VARIABLES_PATH` and list the name in `output-variables` | | Service containers attached to the step | No | Each step that needs one declares it again | ## Artifacts: the way build output moves forward The step-level `artifacts` option lists files or directories that later steps in the same pipeline need. The rules worth knowing: - Paths are **relative to `$BITBUCKET_CLONE_DIR`** and accept glob patterns such as `dist/**`; a pattern that starts with `*` must be quoted, and `.` or `..` are not supported. A file that is not inside the clone directory when the step completes cannot be an artifact. - Later steps **download artifacts by default**. A step that does not need them sets `artifacts: download: false`, and lists any files it produces itself under `paths`. - In a **parallel group**, steps can only use artifacts produced by steps before the group, not by their siblings. The step after the group receives the combination of everything the group produced; if two parallel steps write the same file, the one defined later in the YAML wins. - Artifacts are **retained for at most 14 days** and are **limited to 1 GB each**. A manual deployment or a redeploy that happens after that window has nothing to download and needs a fresh pipeline run. Newer configurations can also name an artifact and mark it `scoped` (kept for that step only, such as logs and screenshots) or `test-report`; the plain glob list remains the common form. ## Caches are not artifacts A **cache** solves a different problem: re-downloading the same third-party dependencies on every run. Pipelines ships predefined caches named after common package managers (plus `docker`), and you can define your own under `definitions: caches`, optionally versioned by `key: files` so the cache is rebuilt when a lock file changes. The documented constraints make caches unsuitable for anything a later step *must* have: - a cache is limited to 1 GB once compressed; - caches expire after 7 days without use and *may be cleared at any time*; - parallel steps can read caches but do not write them; - the documentation's own best practice is "don't cache build outputs, use artifacts instead". A build that breaks when its cache is empty is broken; a deploy step that breaks when its artifact is missing is behaving as designed. ## Reading a two-step configuration In the configuration below, the first step builds into `dist/` and declares it, and the second step finds it already downloaded before its script starts. Remove the `artifacts` block and the second step fails on a missing directory even though both steps ran on the same commit. ```yaml image: acme/build-env:2.1 pipelines: default: - step: name: Build script: - ./build.sh --out dist/ artifacts: - dist/** - step: name: Test the build script: - ./test.sh dist/ ``` The practical consequence for anyone moving an existing build to Pipelines is that the boundary between steps is a hard one. Put work that shares files in the same step, pass outputs forward deliberately as artifacts, and treat every step's start-up (container, clone, artifact download) as a cost that is paid again each time you split the work further.
- How long can a later step or a redeploy rely on an artifact being there?Bitbucket Pipelines keeps artifacts for at most 14 days, and each is limited to 1 GB. A manual deployment step promoted after that window, or a redeploy of an old run, has nothing to download, so the fix is to run a new pipeline that rebuilds the artifact rather than keeping a release waiting on an old one.
- How do you pass a small value, such as a computed version string, from one step to the next?Write `NAME=value` lines to the file at `$BITBUCKET_PIPELINES_VARIABLES_PATH` and list the names under the step's `output-variables`; subsequent steps then see them as variables. The mechanism is capped at 50 variables and 100 KB per pipeline, so it suits version strings and identifiers, not files.
- Why does splitting one long step into many short ones slow some pipelines down?Every step is a new container: it pulls or starts its image, clones the repository again, downloads the artifacts it needs and restores its caches before the script runs. Splitting work multiplies that start-up, so steps should be split where isolation or parallelism pays for it, not by default.
Each step is a hotel room that is cleaned between guests: anything left behind is gone, unless you pack it in your luggage, which is what declaring an artifact does.
- All steps in one pipeline share the same container and workspace.
- Caches are the right place for build output a later step needs.
- Any file on the container's disk can be listed as an artifact.
- Variables exported in one step's script carry over to the next step.
- Artifacts are kept forever, so any old run can be redeployed.