What does the `volumeClaimTemplates` field on a Kubernetes StatefulSet do, and how does it differ from listing a PersistentVolumeClaim under a Deployment's pod volumes?

## Two ways a pod gets a PVC In a plain pod spec you attach storage by naming an existing claim: ```yaml volumes: - name: data persistentVolumeClaim: claimName: shared-data ``` Every replica of a Deployment using that spec points at the single claim `shared-data`. With a ReadWriteMany filesystem that is a legitimate pattern - a shared media directory, say. With ReadWriteOnce block storage it fails as soon as replicas land on different nodes, because the volume can only be attached to one node at a time. And for a database it is simply wrong: three replicas writing one filesystem corrupts it. ## What volumeClaimTemplates changes `spec.volumeClaimTemplates` is a list of PVC specs. The StatefulSet controller creates a real PVC per replica, with the deterministic name: ``` <volumeClaimTemplate.name>-<statefulSetName>-<ordinal> ``` So `data` + `pg` + replica 2 = `data-pg-2`. The container mounts it using the template's name as the volume name - no `volumes:` entry is needed, the controller wires it in. Each claim is provisioned through its StorageClass just like a hand-written PVC, so all the usual mechanics apply: dynamic provisioning, access modes, topology-aware binding, expansion if the class allows it. ## Ownership and lifetime This is the part interviews probe. The generated PVCs are **not** garbage-collected with the pod. When pod `pg-1` is deleted - a rolling update, a node drain, an eviction - the replacement pod is created with the same name and the same ordinal, and it binds the same PVC `data-pg-1`. The data survives. By default the PVCs also outlive the StatefulSet itself: `kubectl delete statefulset pg` removes the pods but leaves `data-pg-0..2` and their PersistentVolumes. That default is deliberate - it makes accidental deletion recoverable - and it is also a common source of orphaned disks nobody is paying attention to. Newer Kubernetes exposes `persistentVolumeClaimRetentionPolicy` to make deletion and scale-down behaviour explicit. ## Why identity and storage go together A StatefulSet gives each replica three stable things: a name (`pg-0`), a DNS record via the headless Service (`pg-0.pg.default.svc`), and now a volume. Together these mean a peer can be addressed and its data found again after any reschedule. Distributed databases rely on exactly this: a Cassandra or etcd or Kafka node re-joining the cluster expects to find its own commit log where it left it, not an empty disk or a peer's data. ## Practical rules - One template per independent volume: many databases use two, `data` and `wal`/`logs`, on different classes. - The StorageClass usually wants `volumeBindingMode: WaitForFirstConsumer` so each zonal disk is created in the zone where its pod can actually be scheduled. - `volumeClaimTemplates` is effectively immutable on a live StatefulSet; changing size or class is not a simple `kubectl edit`. - Sizing is per replica, so 5 replicas x 500 GiB is 2.5 TiB, and scaling up provisions more. - If you genuinely want replicas to share one volume, do not use a template - reference a shared ReadWriteMany claim in the pod spec instead. ## A quick check After creating a 3-replica StatefulSet, `kubectl get pvc` shows three claims with ordinal-suffixed names, each `Bound` to its own PV. If you instead see one claim shared by all pods, someone put a `persistentVolumeClaim` under `volumes:` rather than using a template - a common and consequential mistake.