In DVC, why does a fresh `git clone` of a project lack its tracked data, and how do `dvc push` and `dvc pull` move it?
answer
- Git carries pointers, not bytes
- a registered storage location
- cache to remote, remote to cache
- pull is fetch plus checkout
- status with --cloud before pushing
basics
~20 sGit carries only the small .dvc and dvc.lock pointer files; the bytes stay in each machine's DVC cache. dvc push uploads the cached objects a remote lacks; dvc pull downloads them and places them in the workspace.
solid answer
~40 sA clone brings code, `.dvc` files and `dvc.lock`, which record each output's path and content hash, but the data lives in the cache (`.dvc/cache` by default), which Git ignores. A DVC remote, registered with `dvc remote add -d ci-s3 s3://example-lidar-scans/dvc` and committed in `.dvc/config`, is where those bytes are shared. `dvc push` finds the outputs the current metafiles reference, works out which cached objects the remote is missing and uploads only those; it never uploads code or metafiles, which `git push` handles. `dvc pull` is `dvc fetch` (remote to cache) followed by `dvc checkout` (cache to workspace). `dvc status --cloud` shows what the remote lacks before you push.
code
bash · 11 lines# once, in the project
dvc remote add -d ci-s3 s3://example-lidar-scans/dvc
git add .dvc/config && git commit -m "Add DVC remote"
# after tracking new data
dvc status --cloud # lists objects the remote lacks as new
dvc push # uploads only those objects
git push # code and pointer files travel separately
# on a fresh clone
dvc pull # fetch into the cache, then checkout into the workspacego deeper
Recall the split: Git moves code and pointer files, a DVC remote holds the bytes. Know that push uploads from the cache and pull downloads and places data in the workspace.
Explain pull as fetch plus checkout, how the remote is chosen (output field, -r, core.remote), and what new, deleted and missing mean in dvc status --cloud.
Show you can diagnose a CI pull that finds missing objects, and guard against it with status checks or the pre-push hook from dvc install rather than reminders.
Weigh where the team's remotes live, which machines may write to them, and how push discipline is enforced across many repositories sharing one bucket.
## What a clone brings and what it leaves behind In a DVC project, Git versions the code and the small **metafiles**: `.dvc` files and `dvc.lock`, which record each tracked output's path and content hash. The bytes themselves live in the project's **cache** (by default `.dvc/cache`), and DVC keeps that directory out of Git. A teammate who runs `git clone` therefore gets every pointer and none of the data. Something else has to move the bytes between machines, and that is a **DVC remote**. - A remote is a storage location registered in DVC configuration: an S3, Google Cloud Storage or Azure Blob container, an SSH server, a WebDAV endpoint, or a plain directory such as a mounted network share. - DVC does not provide or recommend a storage service; the team brings its own. - The remote is laid out like the cache, by content hash, so it is not a browsable copy of the workspace. ## Registering a remote ```bash dvc remote add -d ci-s3 s3://example-lidar-scans/dvc git add .dvc/config && git commit -m "Add DVC remote" ``` `dvc remote add` writes a `remote` section to `.dvc/config`; `-d` / `--default` also sets `core.remote`, the remote every data command uses when none is named. The URL's scheme decides the storage type: `s3://`, `gs://`, `azure://`, `ssh://`, `webdav://` or `webdavs://`, or a local or mounted path. An S3-compatible server keeps the `s3://` URL and adds an `endpointurl`. A pip install may need the matching extra, such as `dvc[s3]`. Committing `.dvc/config` is what tells every clone where the data lives. ## dvc push: from your cache to the remote Without targets, `dvc push` collects the outputs referenced by the current `dvc.yaml` and `.dvc` files, looks up their hashes in the cache, asks the remote which objects it is missing, and uploads only those. It does not upload code, `dvc.yaml` or `.dvc` files; that is `git push`'s job. Useful options: - `-r <name>` pushes to a named remote instead of the default. - `-j <number>` sets parallelism; the default is `4 * cpu_count()`, and `dvc remote modify <name> jobs` changes it per remote. - `-a`, `-T` and `-A` widen the scope to all branches, all tags or all commits. The remote a command uses is resolved in this order: 1. a `remote` field written on the output in `dvc.yaml` or the `.dvc` file; 2. the `-r` / `--remote` option; 3. `core.remote`, the default set by `dvc remote add -d` or `dvc remote default`. With none of these set, the command stops with a "no remote specified" error that suggests `dvc remote default` or `-r`. ## dvc pull: fetch, then checkout `dvc pull` has the same effect as `dvc fetch` followed by `dvc checkout`. The fetch half downloads missing objects from the remote into the cache; the checkout half places them in the workspace, linking or copying according to the cache's link-type setting. `dvc fetch` on its own is useful when you want data for several branches in the cache without touching the workspace. Targets (a path, a `.dvc` file or a stage name) limit what is pulled, which matters on a dataset of millions of files. `--allow-missing` lets a pull succeed when some objects are absent from the remote, which is useful in CI jobs that need only part of the data but hides real gaps if used everywhere. ## Seeing what a remote lacks `dvc status --cloud` (`-c`) compares the cache with the default remote; `-r <name>` compares with another one and implies `--cloud`. When nothing differs it prints that the cache and the remote are in sync. Otherwise it lists states: | State | Meaning | Fix | |---|---|---| | `new` | in the local cache, not in the remote | `dvc push` | | `deleted` | in the remote, not in the local cache | `dvc pull` or `dvc fetch` | | `missing` | in neither the cache nor the remote | regenerate it, then push | ## The classic failure A teammate commits a new `lidar-scans.dvc`, runs `git push`, and forgets `dvc push`. CI clones the commit and `dvc pull` reports the files as missing, because the pointer arrived and the bytes never left the laptop. The fix is a `dvc push` from the machine whose cache holds them; the guard is running `dvc status --cloud` before pushing code, or `dvc install`, whose `pre-push` Git hook runs `dvc push` whenever `git push` runs.
- What does `dvc fetch` do that `dvc pull` does not, and when would you use it?`dvc fetch` downloads objects from the remote into the cache only; `dvc pull` is fetch plus `dvc checkout` into the workspace. Fetch is handy after a fresh clone when you want several branches' or tags' data cached (`-a`, `-T`), before comparing metrics across commits, or when you want to keep the workspace untouched.
- CI's `dvc pull` reports files missing that a teammate has on a laptop. What happened, and how do you prevent it?The teammate committed and pushed the pointer with Git but never ran `dvc push`, so the remote never received the objects. Run `dvc push` from the machine that has them. To prevent it, check `dvc status --cloud` before `git push`, or run `dvc install`, whose `pre-push` hook runs `dvc push` on every `git push`.
- A project has an SSH remote and an S3 remote. Which one does a bare `dvc push` use?An output's own `remote` field wins first, then `-r`, then `core.remote`. If no output names a remote and no default is set, `dvc push` stops with a "no remote specified" error suggesting `dvc remote default` or `-r`; it never picks one of the two on its own.
- git push uploads the DVC-tracked data together with the pointer files
- dvc push also uploads the code and .dvc files to the remote
- dvc pull only fills the cache and leaves the workspace untouched
- dvc push re-uploads every tracked file each time it runs
- DVC provides a hosted default remote, so dvc remote add is optional