skip to content

Cluster Hardening Baseline

7 roadmaps5 questions

A defensible default posture: anonymous auth off, Node plus RBAC authorization, encryption at rest configured, the kubelet's own API authenticated rather than open, NodeRestriction on. 'How would you harden this cluster' expects flags, not adjectives.

part ofKubernetesoverview, primer and where to startread it →
on this pageshow

explore

questions

5

When asked to harden a self-managed Kubernetes kube-apiserver, which flags would you set, and what does each one close?

level: middleimportance: must knowfreq 60%

answer

  1. front door has three stages
  2. binary defaults are permissive
  3. AlwaysAllow hides behind no flag
  4. NodeRestriction is not default-on
  5. health probes ride on anonymous

basics

~10 s

Set --anonymous-auth=false, --authorization-mode=Node,RBAC (the built-in default is AlwaysAllow), --enable-admission-plugins including NodeRestriction, --encryption-provider-config for Secrets at rest, and TLS client flags toward etcd and the kubelets.

solid answer

~40 s

I answer with flags, not adjectives. On kube-apiserver: `--anonymous-auth=false`, so a request with no credentials is rejected instead of running as `system:anonymous`. `--authorization-mode=Node,RBAC`, because a bare binary with no authorization flags and no `--authorization-config` falls back to `AlwaysAllow`. `--enable-admission-plugins=NodeRestriction` (plus whatever else you need), so a kubelet can only modify its own Node and its own pods. `--encryption-provider-config`, so Secrets are not stored in etcd as plaintext. `--etcd-cafile`/`--etcd-certfile`/`--etcd-keyfile`, so etcd is reached over mutual TLS. `--kubelet-certificate-authority`, so the API server checks the kubelet's serving certificate. I would also set `--profiling=false`. Then I check each flag with kube-bench instead of assuming it is set.

code

bash · 2 lines
bash
ps -o args= -C kube-apiserver | tr ' ' '\n' | grep -E -- '--(anonymous-auth|authorization-mode|enable-admission-plugins|encryption-provider-config|kubelet-certificate-authority|profiling)'
kubectl auth can-i --list --as=system:anonymous

go deeper

for a junior

Remember the three stages every API request passes: authentication, authorization, admission. Also remember that anonymous access and AlwaysAllow are the two settings that make them meaningless.

for a middle

Name each flag with its hardened value and its built-in default, and explain which of them kubeadm already sets and which it leaves open.

for a senior

Show that you roll flag changes across control-plane nodes one at a time, and that you know which changes break things, such as unauthenticated health probes, before you make them.

for a principal

Frame the baseline as a measured, re-checked state rather than a one-time edit. Decide which gaps are fixed in the installer's config and which get monitoring, so upgrades do not quietly undo them.

## Why the question wants flags **kube-apiserver** is the one front door to the cluster. Every `kubectl` command, controller and kubelet talks to it, and it is the only component that writes to **etcd**, the cluster's key-value store. Hardening it means changing how it **authenticates** (works out who is calling), **authorizes** (decides whether that caller may do this) and **admits** (checks or changes the object before it is stored). An interviewer who asks "how would you harden this cluster" wants the exact switches. "Lock it down" and "follow best practice" score nothing. A useful point to make early: **the binary's own defaults are permissive**. Installers such as kubeadm override some of them, but not all. On a self-managed cluster you have to read the static-pod manifests in `/etc/kubernetes/manifests` and see what is actually set. ## The baseline, flag by flag | Flag | Built-in default | Hardened value | What it closes | |---|---|---|---| | `--anonymous-auth` | `true` | `false` (or a scoped `AuthenticationConfiguration`) | Requests with no credentials run as `system:anonymous` | | `--authorization-mode` | `AlwaysAllow` when no `--authorization-config` is given | `Node,RBAC` | Every authenticated caller may do anything | | `--enable-admission-plugins` | a default-on set **without** `NodeRestriction` | add `NodeRestriction` | A kubelet editing other Nodes, their labels, or pods not bound to it | | `--encryption-provider-config` | unset | path to an `EncryptionConfiguration` | Secret values stored in etcd as plaintext | | `--etcd-cafile`, `--etcd-certfile`, `--etcd-keyfile` | unset | the etcd CA and an API-server client cert | Plaintext or unauthenticated traffic to etcd | | `--kubelet-certificate-authority` | unset | the CA that signed the kubelet serving certs | The API server trusting any certificate a kubelet presents | | `--profiling` | `true` | `false` | The pprof debug endpoints on the API server | Some notes on the table: - **Node,RBAC in that order.** The **Node authorizer** answers only for kubelets, meaning users in the `system:nodes` group named `system:node:<nodeName>`. **RBAC** answers for everyone else. Node and RBAC never deny outright: each either allows the request or passes it to the next module, and the first allow wins. Leaving `AlwaysAllow` anywhere in the list therefore allows everything the earlier modules did not. - **kubeadm already sets several of these.** It writes `--enable-admission-plugins=NodeRestriction` and `--authorization-mode=Node,RBAC`, and it configures etcd over mutual TLS. It does **not** turn off anonymous auth, set `--kubelet-certificate-authority`, or configure encryption at rest. Those three are the usual gaps on a kubeadm cluster. - **Encryption at rest** is one flag here. Choosing a provider and re-encrypting existing Secrets is a separate subject. ## The anonymous-auth trap Turning off anonymous auth everywhere can take a healthy control plane down. kubeadm's static-pod **liveness, readiness and startup probes** call `/livez` and `/readyz` over HTTPS with no credentials. Many external load balancers probe `/healthz` the same way. These unauthenticated calls work only because the built-in `system:public-info-viewer` ClusterRole is bound to `system:unauthenticated`. With `--anonymous-auth=false`, the probes get 401 and the kubelet keeps restarting the API server. There are two ways around this: 1. Give probes a credential, or point them at a separate health endpoint that does not need one. 2. Use a structured `AuthenticationConfiguration` file (`--authentication-config`). It can enable anonymous auth **only for listed paths**. This file cannot be combined with `--anonymous-auth` when it configures the anonymous authenticator. ```yaml apiVersion: apiserver.config.k8s.io/v1 kind: AuthenticationConfiguration anonymous: enabled: true conditions: - path: /livez - path: /readyz - path: /healthz ``` With this file, a request with no credentials to any other path is rejected before it reaches authorization. A stray ClusterRoleBinding to `system:unauthenticated` can then no longer expose anything except those health endpoints. ## Applying it on a real control plane On a 3-control-plane, 27-worker self-managed cluster you change **one API server at a time**. The kubelet restarts that static pod when its manifest changes, and the load balancer sends traffic to the other two while it restarts. Check each change before moving to the next: 1. Read the current flags (`ps` on the host, or the manifest file) and record the gaps. 2. Change one flag on one control-plane node. Watch the probes and the API server log. 3. Confirm with `kubectl auth can-i --list --as=system:anonymous` and with a kube-bench control-plane run. 4. Repeat on the other two nodes. ## What the flags do not cover The API server flags are half of the baseline. The **kubelet's own API**, **etcd's listeners** and the **workloads' Pod Security** each have their own settings. A cluster whose API server is fully hardened can still be taken over through an unauthenticated kubelet on port 10250.

  • What happens if you set --anonymous-auth=false on a kubeadm control plane and change nothing else?
    kubeadm's static-pod probes call `/livez` and `/readyz` with no credentials. They start getting 401, so the kubelet marks the API server unhealthy and restarts it again and again. Load-balancer health checks on `/healthz` fail the same way. The fix is either to give probes credentials, or to use an `AuthenticationConfiguration` that allows anonymous requests only on the health paths.
  • Why is AlwaysAllow dangerous even when it is listed after RBAC?
    Kubernetes authorizers are asked in order, and the first one to allow the request decides. RBAC never denies outright; when no binding matches it has no opinion, so the request goes on to `AlwaysAllow`, which allows it. Every authenticated identity effectively becomes an administrator.
  • How do you confirm the baseline actually holds instead of trusting the manifest you edited?
    Read the running process arguments, not only the file. Run kube-bench's control-plane and node targets. Test behaviour directly: an unauthenticated `curl` to a non-health path should get 401, and `kubectl auth can-i --as` should show what low-privilege identities can do. A hand-edited manifest that has a typo or has been overwritten by a later upgrade shows up only this way.

Hardening the API server is like re-keying a building's front door: you change who gets in without a badge, which badges open which rooms, and whether deliveries are inspected. None of it secures the side doors, which here are the kubelets and etcd.

saying these in an interview costs you the question

  • Says the API server denies everything by default without any authorization flags
  • Believes NodeRestriction is enabled on every kube-apiserver out of the box
  • Answers only with 'use RBAC' and names no flags
  • Keeps AlwaysAllow in --authorization-mode as a safety fallback
  • Disables anonymous auth everywhere without checking unauthenticated health probes
  • Thinks hardening the API server also secures the kubelet port
open as a page

Why is an open Kubernetes kubelet API on port 10250 a node-takeover risk, and which kubelet settings lock it down?

level: middleimportance: should knowfreq 48%

basics

~20 s

The kubelet API can run commands in any container on its node and read its logs, so anonymous access there means code execution. Disable anonymous auth, enable webhook authentication and Webhook authorization, and set readOnlyPort to 0.

open as a page

A scan finds etcd's client port 2379 on your Kubernetes control-plane nodes reachable from the worker subnet. What is at risk, and how do you lock it down?

level: seniorimportance: should knowfreq 42%

basics

~20 s

Direct etcd access bypasses Kubernetes authentication, RBAC and admission, so a caller can read or rewrite every object, including Secrets. Require client and peer certificates signed by a dedicated etcd CA, and firewall 2379 and 2380 to control-plane hosts only.

open as a page

You inherit a 3-control-plane, 27-worker self-managed Kubernetes cluster that fails many kube-bench control-plane and kubelet checks. How do you plan and sequence hardening without causing an outage?

level: principalimportance: should knowfreq 34%

basics

~20 s

Rank kube-bench failures by what an attacker gains, find who depends on each open setting before closing it, roll changes one control-plane node and small kubelet waves at a time, and bake the result into provisioning.

open as a page

With the Kubernetes Node authorizer enabled, what does the NodeRestriction admission plugin additionally enforce, and which compromised-node attack does it blunt?

level: seniorimportance: nice to knowfreq 28%

basics

~20 s

NodeRestriction checks the content of a kubelet's writes. A kubelet may change only its own Node and pods bound to it, may not change its taints, and may not set node-restriction.kubernetes.io/ labels, so a stolen node cannot pull sensitive workloads onto itself.

open as a page