A tenant's PDF-invoice renderer parses untrusted uploads in a shared 80-node Kubernetes cluster. What does moving it to a dedicated tainted node pool isolate, and what does it still share?

## The scenario The `invoices` tenant runs a **PDF-invoice renderer** that opens files uploaded by the public. Document parsers are a classic source of memory-corruption bugs, and a container is only kernel-level isolation. If the renderer is exploited and the attacker breaks out of the container, they are on the node. In a shared 80-node cluster that autoscales between 20 and 80 nodes, that node also runs other tenants' pods. This answer covers what a **dedicated, tainted node pool** buys, and what it cannot. ## What a dedicated pool isolates A taint on the pool keeps other tenants' pods off it, and node affinity keeps the renderer from landing elsewhere. Together they give: - **No shared kernel with other tenants.** An escape reaches only `invoices` workloads. - **Contained node credentials.** With the **Node authorizer** and the **`NodeRestriction`** admission plugin, which kubeadm enables, a kubelet's credential can `get` only the Secrets, ConfigMaps and volumes referenced by pods bound to that node, and can modify only its own Node object and its own pods. A compromised node leaks this tenant's Secrets, not everyone's. - **No noisy neighbours** on disk I/O, PIDs, conntrack entries or local ephemeral storage. - **Separate capacity accounting.** The pool can be sized and billed per tenant. ## What it still shares | Shared surface | Consequence | |---|---| | kube-apiserver and etcd | An API abuse or request storm still affects all tenants. API Priority and Fairness queues per user but shares each priority level | | Cluster-scoped objects | One set of CRDs, StorageClasses, webhooks and ClusterRoles for everyone | | Cluster DNS | Names in every namespace still resolve from the renderer | | Platform DaemonSets | Log shippers and CNI agents run on the dedicated nodes too, often privileged, and are part of the trust boundary | | Pod network | Still one network, so the default-deny NetworkPolicy is still required | This is why dedicated nodes are often described as **strong soft tenancy**. Only a separate control plane removes the first three rows. ## Making placement enforceable A taint only keeps pods *off* a node until someone writes a matching toleration. RBAC authorizes **verbs on resources**, not the **values of fields**, so any tenant allowed to create pods can add any toleration. Enforce placement at admission: 1. **`PodTolerationRestriction`**, an API server admission plugin that is off by default and enabled with `--enable-admission-plugins`. It reads the namespace annotations `scheduler.alpha.kubernetes.io/defaultTolerations` and `scheduler.alpha.kubernetes.io/tolerationsWhitelist`, adds the defaults and rejects pods whose tolerations fall outside the allowed list. 2. **`PodNodeSelector`**, also off by default. It merges the namespace annotation `scheduler.alpha.kubernetes.io/node-selector` into each pod and rejects a pod whose own node selector conflicts with it. 3. **A policy engine** can express the same rules when you would rather not enable those plugins. ## Defence in depth on the node - A **`RuntimeClass`** (`node.k8s.io/v1`) whose `handler` points at a sandboxed runtime puts a stronger boundary than namespaces and cgroups around the renderer. Its `scheduling.nodeSelector` keeps those pods on nodes that have the handler. - **Pod user namespaces** via `hostUsers: false` map container root to an unprivileged host UID. This needs runtime and kernel support. - The **`restricted` Pod Security Standard** on the namespace. ## Costs to weigh - **Fragmented capacity.** Spare room in the dedicated pool cannot absorb other tenants' load. If Cluster Autoscaler scales the pool, set a sensible minimum, or the renderer waits for a cold node. - **More node groups to patch and upgrade.** - **A false sense of completeness.** If the requirement is "no shared control plane", this is the wrong tool, and the answer is a separate cluster.