When a Kubernetes kubelet must reclaim memory on a node, in what order does it choose which pods to kill? Explain how a pod's Quality of Service class and its resource requests determine that ranking.

## Quality of Service classes Kubernetes assigns each pod one of three **QoS classes** at admission time, purely from its containers' `resources.requests` and `resources.limits`: - **Guaranteed** — *every* container in the pod specifies both CPU and memory requests and limits, and for each resource the limit equals the request. This is the strongest class. - **Burstable** — the pod does not qualify as Guaranteed, but at least one container sets a request or limit. The pod is entitled to its request and may burst above it if the node has spare capacity. - **BestEffort** — no container sets any request or limit. The pod is entitled to nothing. You can read the assigned class with `kubectl get pod X -o jsonpath='{.status.qosClass}'`. ## What "request" means for eviction A **request** is a reservation: the scheduler only places a pod on a node whose unallocated capacity covers it, and the kubelet treats it as the pod's entitlement. A **limit** is a ceiling enforced by cgroups: exceed the memory limit and the container is `OOMKilled` individually. Eviction ranking is about the request, not the limit. The kubelet asks: *is this pod using more memory than it reserved?* ## The ranking algorithm When a memory signal breaches its threshold, the kubelet builds a list of candidate pods and sorts them by, in order: 1. **Usage relative to request.** Pods whose memory working set is *above* their memory request rank ahead of pods below their request. This is the dominant term. 2. **Pod priority.** Among pods in the same over/under bucket, the one with the lower `priorityClassName` value is evicted first. This is how you protect critical infrastructure DaemonSets — give them a high PriorityClass such as `system-node-critical`. 3. **Magnitude of the overage.** Among equal-priority pods that are all over request, the one exceeding its request by the most is evicted first. Because a BestEffort pod has a memory request of zero, *any* memory it uses puts it over request, so BestEffort pods land at the head of the list every time. A Burstable pod that requested 256Mi and is using 900Mi is next. A Burstable pod using 100Mi against a 256Mi request, and a Guaranteed pod (which by construction cannot exceed its request without hitting its equal limit and being OOMKilled first), sit at the bottom. So the familiar shorthand — *BestEffort first, then Burstable, then Guaranteed* — is a **consequence** of the ranking rather than the rule itself. Saying the rule in terms of usage-versus-request is the better answer, because it explains why a Burstable pod comfortably inside its request survives while a greedier one does not. ## Guaranteed is not immunity If reclaiming from every over-request pod still leaves the node below its threshold, the kubelet keeps going into Guaranteed pods. The class buys you an ordering advantage, not an exemption. Similarly, high pod priority delays but does not prevent node-pressure eviction — this is a well-known asymmetry with API-initiated eviction (the `/eviction` subresource used by `kubectl drain`), which *does* respect PodDisruptionBudgets. **Node-pressure eviction ignores PodDisruptionBudgets and does not honour graceful termination on hard thresholds.** ## The kernel side: OOM scores The same classes shape behaviour one level down. The kubelet sets each container's `oom_score_adj`: Guaranteed containers get −997 (essentially never chosen), BestEffort get 1000 (chosen first), and Burstable get a value scaled inversely to their memory request — the smaller the request relative to node capacity, the higher the score. So if the kernel OOM killer fires before the kubelet can act, victim selection follows the same philosophy. ## Practical consequences - **Set memory requests close to real steady-state usage.** That single act moves a pod from "always over request" to "under request" and is the cheapest reliability win available. - **Do not run production workloads BestEffort.** They are the designated victims. - **Use PriorityClasses deliberately** for logging/monitoring agents and other node-critical DaemonSets. - **Guaranteed for latency-critical or stateful pods**, accepting the cost that limit equals request means no bursting and lower packing density. ## Interview framing State the three classes and how they are derived, give the ranking as usage-vs-request → priority → magnitude of overage, note that BestEffort-first falls out of that, and finish with the caveat that Guaranteed is only last in line, not exempt, and that node-pressure eviction ignores PodDisruptionBudgets.