skip to content

How does the Secrets Store CSI Driver deliver external secrets into a Kubernetes Pod, and what does a SecretProviderClass's secretObjects field add?

level: middleimportance: should knowfreq 40%

answer

  1. a volume, not a controller loop
  2. class names provider and parameters
  3. files first, Secret optional
  4. no mount, no synced Secret
  5. rotation flag off by default

basics

~20 s

A Pod mounts an inline CSI volume naming a SecretProviderClass. At mount time the driver asks a provider plugin for the values and writes them as files. secretObjects also mirrors them into a Kubernetes Secret, for example for env vars.

solid answer

~40 s

The **Secrets Store CSI Driver** runs as a DaemonSet, with one provider plugin per secret manager. A Pod declares an inline `csi` volume with `driver: secrets-store.csi.k8s.io` and `volumeAttributes.secretProviderClass`. When the kubelet mounts the volume, the driver reads that **`SecretProviderClass`** (`secrets-store.csi.x-k8s.io/v1`), passes its `parameters` to the provider, and writes the returned objects as files into a tmpfs mount, often authenticating as the Pod's ServiceAccount. By default no `Secret` object is created. **`spec.secretObjects`** asks the driver to also create a Secret (`secretName`, `type`, and `data[]` mapping an `objectName` to a `key`) so the values can feed env vars or other objects. That Secret exists only while at least one Pod mounts the volume. Refreshing values in place requires `--enable-secret-rotation`, which is off by default.

code

yaml · 19 lines
yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: tracking-db
  namespace: shipment-tracking
spec:
  provider: vault
  parameters:
    roleName: tracking-api
    objects: |
      - objectName: db-password
        secretPath: secret/data/tracking/db
        secretKey: password
  secretObjects:
    - secretName: tracking-db-env
      type: Opaque
      data:
        - objectName: db-password
          key: DB_PASSWORD

go deeper

for a junior

Recall that the CSI driver shows secrets as files in a mounted volume, configured by a SecretProviderClass that names a provider.

for a middle

Walk through the mount-time flow, what secretObjects adds, and why the synced Secret exists only while a Pod mounts the volume.

for a senior

Show that you know the failure modes: Pods stuck at mount during a provider outage, missing sync RBAC, and rotation left disabled.

for a principal

Weigh keeping secrets out of etcd and fetching per Pod against start-time dependency on the provider, and choose per workload class rather than per cluster.

## What the driver is The **Container Storage Interface (CSI)** is Kubernetes' plugin interface for volumes. The **Secrets Store CSI Driver** uses it for something other than disks: it presents secrets from an external secret manager as **files in a Pod's volume**. It runs as a **DaemonSet** on every node, next to a **provider plugin** for each backend. The driver handles the Kubernetes side, and the provider knows how to talk to one particular secret manager. ## The pieces - **`SecretProviderClass`** (API group `secrets-store.csi.x-k8s.io`, version `v1`) is namespaced. It contains: - `spec.provider`, which names the provider plugin; - `spec.parameters`, a free-form string map that only the provider interprets (which objects to fetch, which role to use); - `spec.secretObjects`, which optionally mirrors the fetched objects into Kubernetes Secrets. - **The Pod's volume** is an inline `csi` volume with `driver: secrets-store.csi.k8s.io`, `readOnly: true` and `volumeAttributes.secretProviderClass: <name>`. - **`SecretProviderClassPodStatus`** is an internal object the driver writes to record which Pod mounted which class and which object versions it received. ## Mount-time flow 1. The scheduler places the Pod, and the kubelet on that node asks the CSI driver to publish the volume. 2. The driver looks up the `SecretProviderClass` in the Pod's namespace. 3. It calls the provider over a local socket with the parameters and the Pod's identity, typically the Pod's ServiceAccount token, so access can be scoped per workload. 4. The provider fetches the secrets and returns their contents. 5. The driver writes them as files into a tmpfs-backed mount. The container starts and reads them from its `mountPath`. The secret is fetched **at Pod start**. If the provider is unreachable, the mount fails and the Pod stays in `ContainerCreating`, with mount errors in its events. That is a real difference from the External Secrets Operator, where a Secret already exists before the Pod is scheduled. ## secretObjects: syncing to a Kubernetes Secret Some consumers cannot read files: an env var, an Ingress TLS reference, an `imagePullSecrets` entry. For those, `secretObjects` tells the driver to also create a Secret: - `secretName` and `type` (for example `Opaque` or `kubernetes.io/tls`); - `data[]`, where each entry maps an `objectName` (a mounted object) to a `key` in the Secret; - optional `labels` and `annotations`. The caveats are what interviewers look for: - **Sync happens only after a mount.** The Secret is created when a Pod mounts the volume. A Deployment that only uses `secretKeyRef` and never mounts the CSI volume will wait forever for a Secret that never appears. - **Lifetime follows the consumers.** The driver sets owner references from the mounting Pods' owners (for example their ReplicaSet), so the Secret is garbage-collected once nothing that mounts it remains. - **It needs extra RBAC.** The driver must be allowed to write Secrets. The Helm chart gates that behind `syncSecret.enabled`, which is `false` by default. ## Rotation | Driver setting | Default | Effect | |---|---|---| | `--enable-secret-rotation` | `false` | Periodically re-fetches and rewrites the mounted files | | `--rotation-poll-interval` | `2m` | How often that re-fetch runs when rotation is on | With rotation on, the **files** in running Pods are updated, and a synced Secret is updated too. Environment variables built from that Secret still do not change in a running container. The driver's own flag help still labels rotation alpha. ## CSI driver compared with the External Secrets Operator | Aspect | Secrets Store CSI Driver | External Secrets Operator | |---|---|---| | Where values land | Files in the Pod (Secret optional) | A Kubernetes Secret, always | | When they are fetched | At Pod mount, then per poll if rotation is on | On a timer, independent of Pods | | Provider outage at Pod start | New Pods fail to start | New Pods start with the last synced Secret | | Values in etcd | Not unless `secretObjects` is used | Yes, as a Secret | | Runs as | Node DaemonSet plus provider plugins | One cluster controller | Teams that do not want secret values in etcd at all prefer the file-only CSI path. Teams that want Pods to start even during a secret-manager outage prefer ESO.

  • A Deployment uses secretKeyRef to a Secret named in secretObjects, and its Pods are stuck with CreateContainerConfigError. Why?
    The driver creates the synced Secret only when a Pod mounts the CSI volume that references the class. If the Pod spec never mounts that volume, the Secret never appears and the env reference cannot resolve. Add the `csi` volume and mount it in the same Pod, and make sure the driver was installed with secret-sync RBAC.
  • During a 48-node cluster upgrade, drained Pods fail to reschedule because the secret manager is briefly unreachable. Would the External Secrets Operator have behaved differently?
    Yes. The CSI driver fetches at mount, so every rescheduled Pod depends on the provider at that moment. ESO keeps a synced Secret in the cluster, so Pods start from it even while the provider is down. The trade is that ESO stores the value in etcd.

saying these in an interview costs you the question

  • The CSI driver always creates a Kubernetes Secret
  • secretObjects syncs even if no Pod mounts the volume
  • Mounted files refresh automatically with no driver flag
  • Rotation also updates environment variables in running containers
  • SecretProviderClass parameters mean the same for every provider