In Kubernetes, after a pod is deleted and its Deployment replaces it, can you still read the old pod's container logs with kubectl, and why?

## Where Kubernetes keeps container logs Kubernetes itself has **no central log store**. A container writes to **stdout** and **stderr**; the **container runtime** (containerd or CRI-O, behind the **CRI**) captures both streams and writes them to a file on the node's disk. The **kubelet** — the node agent — owns the directory layout and the rotation of those files. When you run `kubectl logs <pod>`: 1. `kubectl` calls the pod's `log` subresource on **kube-apiserver**. 2. The API server looks up which node runs the pod and proxies the request to that node's kubelet. 3. The kubelet opens the container's current log file and streams it back. Nothing about that path touches etcd. Log text is never part of the pod object, so it cannot outlive the files on the node. ## What survives what | Event | Pod object | Log files on the node | What `kubectl logs` gives you | |---|---|---|---| | Container restarts in place | same pod | new file for the new instance; the previous instance's file is kept | current instance; `--previous` gives the one before | | Pod deleted (rollout, scale-down, `kubectl delete`) | gone | removed by the kubelet during cleanup | an error — the pod is not found | | Pod evicted for node pressure | kept, phase `Failed` | may be removed, because eviction tells the kubelet to reclaim the pod's resources | do not count on anything | | Node deleted or its disk replaced | pods rescheduled elsewhere | gone with the disk | nothing | Two details matter here: - **Restarts keep only one old instance.** By default the kubelet's container garbage collection keeps one dead instance per container, which is why `--previous` means "the instance just before the current one", not "any earlier run". - **A replacement pod is a different pod.** A Deployment's ReplicaSet creates a new pod with a new name and UID. It has no link to the deleted pod's files, and `kubectl` does not follow a Deployment to its old pods. ## Why deletion wipes the evidence Once a pod is deleted, the kubelet stops its containers and then cleans up everything it created for that pod: the containers themselves, the sandbox, and the pod's log directory under `/var/log/pods/`. That is deliberate — a node that kept every dead pod's logs would fill its disk. The consequence is that **deleting a failing pod is a destructive act for debugging**, even though the controller replaces it within seconds. The same applies to things that delete pods for you: - a **rolling update** scales the old ReplicaSet down, deleting its pods; - **`kubectl rollout undo`** does the same to the bad revision's pods; - **scaling to zero** deletes every pod; - a CronJob's **history limits** or a Job's **`ttlSecondsAfterFinished`** delete finished Jobs, and their pods and logs with them. ## Keeping the evidence A practical order of work before you remove a broken pod: 1. Save the current output: `kubectl logs <pod> -c <container> > current.log`. 2. If the container has restarted, save the previous instance: `kubectl logs <pod> -c <container> --previous > previous.log`. 3. Save `kubectl describe pod <pod>` — it includes recent Events and the container's last termination state, and Events expire on their own. 4. Only then delete, roll back or scale. For anything you cannot capture by hand — a pod that died at 3 a.m. — you need a **node-level log collector** that tails the files and ships them to a durable store before the kubelet removes them. How that pipeline works is a separate subject; the Kubernetes part is only that the files exist, where they are, and for how long. ## A worked example On a single-node development cluster on a laptop, a video-transcoding worker pod `transcoder-7f9c4d6b8-x2lqp` in namespace `media` starts failing on one input file. The developer runs `kubectl rollout restart deployment/transcoder -n media` to "clear it". The Deployment creates `transcoder-7f9c4d6b8-q4k2v`, deletes the old pod, and `kubectl logs transcoder-7f9c4d6b8-x2lqp -n media` now returns a not-found error. The encoder's stack trace existed only in the deleted pod's log file, and that file is gone. Had the developer run `kubectl logs --previous` first, the trace would be on their disk. ## Ephemeral debug containers are no exception A container added with `kubectl debug` lives in the pod's `spec.ephemeralContainers`. Its output is written to the same per-pod log directory and read with `kubectl logs <pod> -c <debug-container-name>` — and it disappears with the pod in exactly the same way.