A teammate runs `kubectl get pods` and sees several pods with the status `Evicted`. What does that status mean in Kubernetes, what typically causes it, and what would you check first?

## What eviction is Every Kubernetes node runs a **kubelet**, the agent that starts and supervises containers. The kubelet is also responsible for keeping the node itself alive. If a node runs completely out of memory or disk, the Linux kernel starts killing processes arbitrarily — possibly the kubelet, the container runtime, or an SSH daemon — and the node becomes unusable. To avoid that, the kubelet watches a set of resource signals and, before the node hits the wall, proactively kills pods. That proactive killing is called **node-pressure eviction**. An evicted pod ends up in phase `Failed` with `reason: Evicted` and a human-readable message such as `The node was low on resource: memory. Container api was using 1200Mi, which exceeds its request of 256Mi.` ## Why the object sticks around Kubernetes deliberately leaves the evicted pod object in the API as a record — otherwise the eviction would be invisible after the fact. This is why a cluster with a chronic memory problem accumulates dozens of `Evicted` pods. They consume no CPU or memory (no containers are running); they are just API records. The kubelet and the pod garbage collector clean them up once the count passes a threshold, and you can remove them manually: ``` kubectl delete pods --field-selector status.phase=Failed ``` Deleting them fixes nothing — it only tidies output. The actual fix is on the node or in the pod's resource requests. ## What actually gets restarted Eviction deletes the pod's containers, not the workload. If the pod is owned by a Deployment/ReplicaSet, StatefulSet, or Job, the controller notices the replica count is short and creates a **new pod**, which the scheduler places on some node — possibly the same one, if the pressure has cleared. A bare pod (created directly, with no controller) is *not* recreated; it just stays `Evicted`. This is one practical reason to run workloads through controllers. A DaemonSet pod will be recreated on the same node, since that is where it belongs, which is why DaemonSets often show repeated eviction churn on a sick node. ## The common causes - **Memory**: the sum of what pods actually use approaches the node's capacity. Pods that use far more than they requested are the first candidates. - **Disk / ephemeral storage**: logs, images, or files written inside containers or into `emptyDir` volumes fill the node's filesystem, raising `DiskPressure`. - **Inodes**: rarer, but a node can exhaust inodes while still showing free bytes. Crucially, eviction is a *node-level* symptom. Seeing pod A evicted does not mean pod A is the buggy one — the kubelet picks victims by a ranking (Quality of Service class and how far usage exceeds requests), so a well-behaved but low-priority pod can be evicted because a different pod ate the node. ## Distinguishing it from OOMKilled Candidates conflate two different kills: - **`OOMKilled`** — a *container* exceeded its own memory **limit** (or the node kernel OOM killer hit it). The pod stays `Running` and the container restarts; you see it in `kubectl describe pod` under Last State, with restart count climbing. - **`Evicted`** — the *kubelet* removed the whole pod because the **node** was short on a resource. The pod is gone, not restarted. ## First moves in an interview answer Describe the pod to read the eviction message, describe the node to confirm which condition flipped (`MemoryPressure`, `DiskPressure`), check `kubectl top nodes`/`top pods` (or your metrics stack) for who is consuming, and then decide: raise requests so the scheduler stops overpacking the node, fix a leak, add a memory limit, or add capacity. Mentioning that you would also check whether requests are realistic — rather than only blaming the node — is what separates a rote answer from a useful one.