How does kubectl cp move files between your machine and a container, and why does it sometimes fail with an error mentioning tar?
answer
- cp = exec + tar stream
- no tar in image = failure
- exec cat as the fallback
- namespace/pod:path, -c container
- bytes cross the API server
basics
~20 skubectl cp is exec plus tar: it runs tar inside the container and streams the archive over the exec channel. If the image has no tar binary — distroless or scratch — the copy fails. Fall back to exec with cat, or a debug container.
solid answer
~50 s`kubectl cp prod/mypod:/var/log/app.log ./app.log -c app` copies out; reversing the arguments copies in. There is no dedicated file-transfer API: kubectl execs `tar` inside the container and pipes the archive through the same streaming channel `kubectl exec` uses, so it needs `create` on `pods/exec` **and** a `tar` binary in the image. That is the usual failure: minimal images have no tar, and you get an error about tar not being found. Workarounds: stream a single file out with `kubectl exec mypod -- cat /path > local`, push one in with `kubectl exec -i mypod -- sh -c 'cat >/path' < local` (still needs a shell), or attach an ephemeral debug container with `--target` and read the app's files through `/proc/1/root/`. Other gotchas: relative paths resolve against the container's working directory, symlinks are not followed, the whole stream crosses the API server so large files are slow and rude, and old kubectl versions had path-traversal vulnerabilities in the extraction step.
code
bash · 3 lineskubectl cp prod/mypod:/var/log/app.log ./app.log -c app
kubectl cp ./patch.yaml prod/mypod:/tmp/patch.yaml -c app
kubectl exec -n prod mypod -c app -- cat /etc/app/config.yaml > config.yamlgo deeper
Know the syntax in both directions and that -c picks the container.
Explain the exec-plus-tar implementation, the tar dependency, and the cat-based fallback.
Add the control-plane cost, read-only filesystem and ephemerality consequences, and why artifacts should be written to storage by the app instead.
Design the artifact path — heap dumps and profiles to object storage or a sidecar — so routine incident work never needs manual copying out of pods.
## What it really is Kubernetes has no file-transfer endpoint. `kubectl cp` is a client-side convenience built on `kubectl exec`: - **Copy out**: kubectl execs `tar cf - <path>` inside the container and extracts the resulting stream locally. - **Copy in**: kubectl tars the local path on your machine and execs `tar xf -` inside the container, feeding the archive to its stdin. Everything therefore inherits exec's properties: it goes through the API server to the kubelet to the runtime, it is authorized by `create` on `pods/exec`, it appears in the audit log, and it only works on a **running** container. ## The tar requirement Because both directions shell out to `tar` inside the container, the image must contain a `tar` binary on `PATH`. Distroless, `scratch`, and many hardened images do not, so the command fails with a message about tar not being found or an unexpected EOF. Some BusyBox tar builds also lack options kubectl expects, producing confusing partial copies. Alternatives when tar is absent: - Single file out: `kubectl exec mypod -c app -- cat /etc/app/config.yaml > config.yaml`. Works if `cat` exists; for truly empty images even that fails. - Single file in: `kubectl exec -i mypod -c app -- sh -c 'cat > /tmp/f' < f` — needs a shell. - No binaries at all: attach an ephemeral container with `kubectl debug --target=app`, then read the app's files at `/proc/1/root/...` using the debug image's tools and copy from the debug container instead. - Structural fix: mount an `emptyDir` shared with a sidecar, or have the app write artifacts (heap dumps, profiles) to a volume or object storage rather than requiring hand-copying. ## Syntax and semantics The container path is written `namespace/pod:path`, and `-c` selects the container. Paths inside the container are interpreted relative to the container's working directory when not absolute — a common source of file-not-found confusion, so prefer absolute paths. Directory copies follow tar's semantics, where a trailing slash and whether the destination already exists change whether you get `dest/dir` or `dest/dir/dir`; test with something small before copying something large. Symlinks are not dereferenced. ## Practical cautions - **Volume.** Bytes traverse the API server, a control-plane component. Copying a multi-gigabyte heap dump this way loads the control plane and often times out; prefer writing to object storage from inside the pod. - **Ephemerality.** A file copied into a container lives in the writable layer and is gone at restart. Never use `kubectl cp` to deploy a config fix — that belongs in a ConfigMap or the image. - **Read-only root filesystems** reject copies into most paths; target a mounted `emptyDir` such as `/tmp` instead. - **Security.** Copying *out* means your local tar extraction trusts data produced inside the container; historic kubectl vulnerabilities allowed a malicious container to write files outside the destination directory via crafted paths or symlinks. Keep kubectl current and extract into a scratch directory when handling untrusted workloads.
- A teammate fixes a production bug by kubectl cp-ing a corrected config file into a running pod. What is wrong with that?The file lives only in that container's writable layer, so it disappears on the next restart and no other replica has it, which produces a fleet where replicas behave differently and the fix vanishes unpredictably. It also leaves no record in Git or the API. The change belongs in a ConfigMap or the image, rolled out by the controller.
saying these in an interview costs you the question
- Believing kubectl cp uses a dedicated file-transfer API rather than exec plus tar
- Not knowing why it fails on distroless images
- Using it to patch running pods as a fix
- Streaming very large dumps through the API server
- Assuming it works on a container that is not running