skip to content

For CI on a multi-gigabyte Git repository, how do you choose between shallow, partial, and full clones?

level: principalimportance: should knowfreq 34%

answer

  1. Ask what the job reads, not how fast it clones
  2. Three levers remove three different things
  3. A warm local source beats a clever flag
  4. One failure mode is local, one is network-shaped
  5. Verify the flag was actually honoured

basics

~20 s

Start from what each job needs from history, not from clone speed. Build-only jobs take a shallow single-branch clone; jobs needing tags, blame or merge-base take a partial clone; jobs needing both take a full clone warmed from a local mirror or cache.

solid answer

~50 s

Classify jobs by their dependence on the commit graph. A compile-and-package job needs only the current tree — `git clone --depth 1 --branch <ref> --no-tags` is the cheapest correct answer. A job that derives a version from tags, computes a merge base against the target branch, or reads blame needs the graph but not old file contents — `--filter=blob:none` keeps history while deferring blobs. A job that needs deep history *and* lots of content should clone fully, but from a warm local source: keep a bare mirror on the runner and `git fetch` into the workspace, or use `git clone --reference <mirror> --dissociate`. Then measure: partial clones trade a fast start for lazy round-trips, so a job that touches many old files can be slower than the full clone it replaced, and it becomes unavailable if the server is unreachable mid-job.

code

bash · 8 lines
bash
# build-only job
git clone --depth 1 --branch "$REF" --no-tags --single-branch "$URL" src

# job that needs merge-base and tags
git clone --filter=blob:none --branch "$REF" "$URL" src

# heavy job, warmed from a runner-local mirror
git clone --reference /var/cache/git/app.git --dissociate "$URL" src

go deeper

for a junior

Know the two common CI recipes: a shallow single-branch clone when you only need to build, and a full clone when a step needs history such as tags.

for a middle

Explain which commands each option breaks and be able to justify a specific flag choice by naming the step that requires history or file contents.

for a senior

Diagnose a slow or flaky pipeline by separating checkout time from job time, verify the flags took effect, and introduce a warm local mirror where filtering is the wrong tool.

for a principal

Own the estate-wide policy: a small set of sanctioned recipes per job class, an explicit view of how each shifts the failure mode toward the host, and a way to review the choice as the repository grows.

## Frame it as a requirements question The common failure is picking a clone flag to make one number smaller — checkout duration — without asking what the job reads. Three orthogonal levers exist, and each removes something different: - `--depth` / `--shallow-since` remove **commits**. - `--filter=blob:none` / `--filter=tree:0` remove **object content**, keeping the graph. - `git sparse-checkout` removes **working-tree files**, keeping the index. What you can safely remove follows from what the job walks. ## Classify the jobs **Needs only the current tree.** Compile, unit test, lint, container build, static-site publish. A shallow single-branch clone is correct and dramatically cheaper. Add `--no-tags` if nothing reads tags. **Needs the graph, not the content.** Version stamping from `git describe`, changed-file detection via merge base with the target branch, commit-message policy checks, ownership queries. A partial clone with `--filter=blob:none` preserves all of these, since they walk commits and trees. **Needs graph and content.** Full-history analysis, bisect-driven investigations, large blame sweeps, release tooling that reads old artifacts from history. Clone fully — but warm it. ## Warming beats filtering The cheapest full clone is one that mostly does not transfer. Two mechanisms: - Keep a **bare mirror** on the runner, refresh it periodically, and have jobs `git fetch` into their workspace from it. Transfer becomes the delta since the last refresh. - Use `git clone --reference <local-repo> --dissociate <url>`: Git borrows objects from the local repository during the clone, then copies what it borrowed so the workspace no longer depends on it. Without `--dissociate` the workspace holds a fragile dependency on the reference repo continuing to exist and to keep those objects, which is a classic source of mysteriously broken workspaces after a `git gc`. Any cached mirror needs maintenance — periodic `git gc` or `git repack` — or it degrades into a pile of loose objects and small packs. ## Weigh the failure modes, not just the speed **Shallow** fails deterministically and locally: merge-base and describe simply do not work. That is easy to catch in a first run and hard to catch when a rarely-taken branch of the pipeline hits it months later. **Partial** fails probabilistically and remotely: the job works but pauses for lazy fetches, and every one of those pauses is a dependency on the host being available *for the whole duration of the job*, not just at checkout. On a fleet running thousands of jobs, that changes the blast radius of a host incident: with full clones a host outage stops new jobs, with partial clones it can fail jobs already in flight. **Full** is predictable but expensive, and the expense multiplies by concurrency — it is a fleet-level bandwidth and storage decision, not a per-job one. ## Make it measurable and reviewable Instrument checkout duration and total job duration separately; a clone flag that halves the first while adding to the second is a regression disguised as an optimization. Verify that the filter or depth actually took effect — filtering requires server support, and if it is absent the flag is quietly ignored and you paid for a full clone anyway. Finally, make the choice explicit and few. Two or three sanctioned checkout recipes, each tied to a job class and documented with what it breaks, is far more maintainable than every team tuning `--depth` until something fails in a release job at the worst moment.

  • Why can a partial clone make a CI job slower overall?
    The clone is fast because blobs are deferred, but every command that touches a missing file content triggers a fetch. A job that checks out several revisions, blames long files, or diffs against old commits can issue many round-trips, each with latency. Measure end-to-end job time, not checkout time, before declaring a win.
  • What is the risk of git clone --reference without --dissociate?
    The new repository borrows objects from the reference repository instead of copying them, recorded via an alternates entry. If the reference repo is pruned, garbage collected, or removed, the borrowing repository loses objects and breaks in confusing ways. `--dissociate` copies the borrowed objects at the end, keeping the speed benefit without the coupling.
  • How would you detect that a checkout flag is not doing what you expect?
    Compare measured transfer size and checkout time against a control run, and inspect the resulting repository: a shallow clone has a `.git/shallow` file, and a partial clone sets `remote.origin.promisor` and `remote.origin.partialclonefilter`. If those markers are absent, the server ignored the request and you received a full clone.
  • Should every job in a large estate use the same clone recipe?
    No, but the number of recipes should be small and owned centrally. Two or three documented options tied to job classes give most of the benefit; letting every team hand-tune depth spreads a fragile assumption across pipelines that nobody re-validates when history or workflow changes.

saying these in an interview costs you the question

  • Optimizes checkout time without measuring total job time
  • Applies --depth 1 globally, then patches jobs that break
  • Assumes the filter worked without checking server support
  • Uses --reference without --dissociate on ephemeral caches
  • Ignores that partial clones depend on the host mid-job

context

resolving links…