You need a set of Kubernetes nodes reserved exclusively for one team's workload, so that nothing else lands there and that team's pods always run there. Describe exactly how you would configure it and why a toleration alone is not enough.

## The two-sided nature of placement Kubernetes splits placement control into repulsion and attraction, and dedication needs both: - **Repulsion lives on the node**: taints. Only a taint can prevent pods that know nothing about your pool from landing on it. Node affinity cannot do this, because affinity is opt-in on the pod side and pods that don't declare it are unconstrained. - **Attraction lives on the pod**: `nodeSelector`, node affinity. Only these can make your pods actually prefer or require the pool. Miss the first and other teams' workloads bin-pack onto your expensive GPU nodes. Miss the second and your workload scatters across the general pool while the dedicated nodes sit empty — the classic "I added the toleration and nothing changed" bug report. ## The concrete recipe 1. **Label the nodes** so pods can select them: `kubectl label nodes n1 pool=team-a`. 2. **Taint the nodes** so nothing else can enter: `kubectl taint nodes n1 dedicated=team-a:NoSchedule`. 3. **In the workload's pod template** add both a toleration matching the taint and a `nodeSelector: {pool: team-a}` (or a `requiredDuringSchedulingIgnoredDuringExecution` node affinity if you need set-based matching such as `pool In [team-a, team-a-spare]`). By convention people reuse the same key on both sides (`dedicated=team-a` as label *and* as taint) — the API allows it and it keeps manifests readable, but they are two separate fields with two separate meanings. ## Choosing the effect Use `NoSchedule` for steady-state dedication. `NoExecute` additionally evicts anything already running that doesn't tolerate the taint — the right choice when you are converting an existing shared pool into a dedicated one and want the squatters gone immediately, but understand you are triggering an involuntary disruption that does not respect PodDisruptionBudgets. `PreferNoSchedule` is not dedication at all; it is a scheduling hint and other pods will still land there under pressure. ## Where to apply the taint On managed platforms, set the taint and label in the **node pool / managed node group / machine template** specification so nodes are born with them. If you taint by hand after a node registers, there is a race: between the node becoming Ready and your taint landing, the scheduler can bind arbitrary pending pods to it. That window reappears every time the pool scales out or replaces a node, so hand-tainting is not a stable design — it only looks like it works. ## Do not break the node's own agents A tainted node still needs its log shipper, CNI agent, node exporter and CSI node plugin. DaemonSet pods are given tolerations for the standard condition taints automatically, but **not** for your custom `dedicated` taint — you must add it to those DaemonSets, or use a blanket `operator: Exists` toleration in the agents you control. Forgetting this is how a dedicated pool ends up with no metrics and no logs. ## Making the boundary real As specified so far, dedication is a *convention*: any team that copies the toleration into their manifest gets access to the pool. If the pool exists for cost allocation or compliance, enforce it: - A validating admission policy (Kyverno, Gatekeeper, or a `ValidatingAdmissionPolicy`) that rejects pods carrying the `dedicated=team-a` toleration outside team A's namespaces. - The `PodTolerationRestriction` admission plugin, which can whitelist which tolerations a namespace may use and inject default ones. - ResourceQuotas in the owning namespace so the team cannot exceed the pool it paid for. ## Costs and tradeoffs to raise Exclusive pools fragment cluster capacity. Each pool needs its own headroom for spikes and its own spare node for failure, so N dedicated pools cost meaningfully more than one shared pool of the same total size, and a burst in one pool cannot borrow idle capacity from another. Dedication is justified by hardware (GPUs, local NVMe, high-memory shapes), licensing tied to cores, noisy-neighbour isolation for latency-critical services, or compliance boundaries — not by org-chart tidiness. When you only need a *tendency*, `PreferNoSchedule` plus preferred node affinity gives soft separation while keeping the capacity fungible. ## Verifying After rollout, check `kubectl get pods -o wide` that the workload actually landed on the pool, and check that nothing foreign is there: `kubectl get pods --all-namespaces --field-selector spec.nodeName=n1`. A pending pod's `describe` output will tell you plainly whether the taint or the selector is the blocking predicate.