skip to content

A Kubernetes ConfigMap is rejected because its data exceeds roughly 1 MiB. Where does that limit come from, and what are your options for delivering configuration larger than that to a container?

level: middleimportance: should knowfreq 36%

answer

  1. etcd default max value ~1.5 MiB → ConfigMap cap 1 MiB
  2. data + binaryData counted together, per object
  3. kubelet caches it on every consuming node
  4. split / bake into image / PV / init-container fetch
  5. annotations capped at 256 KiB too

basics

~20 s

The API server caps a ConfigMap's data at about 1 MiB because these objects are stored whole in etcd, which has a default per-value size limit. Options: split the data, bake it into the image, mount a PersistentVolume, or fetch it at startup from object storage or a config service.

solid answer

~60 s

ConfigMaps live in etcd, and etcd enforces a maximum request/value size (1.5 MiB by default), so Kubernetes caps ConfigMap and Secret data at roughly 1 MiB. It is not a tunable per-object setting you should be raising. The limit is a design signal: ConfigMaps are for configuration, not for data. Large payloads also cost more than storage — every kubelet with a consuming Pod watches and caches the object in memory, and etcd replicates every write to all members, so a multi-megabyte, frequently-updated object stresses the control plane. Options when you legitimately exceed it: - **Split** into several ConfigMaps mounted into the same directory (works when the payload is many files). - **Bake it into the image** if it changes at the same cadence as code — the registry is built for large blobs. - **Mount a PersistentVolume** (or a read-only ReadOnlyMany volume) for genuinely large data sets. - **Fetch at startup** with an init container or sidecar from object storage or a config service, keeping the ConfigMap for the pointer and credentials reference.

code

bash · 5 lines
bash
kubectl create configmap big --from-file=./huge.json
# error: ConfigMap "big" is invalid: []: Too long: must have at most 1048576 bytes

# check before you push
wc -c ./huge.json

go deeper

for a junior

Know that the cap is about 1 MiB and comes from etcd, and that large data belongs in the image or a volume instead.

for a middle

Explain that data plus binaryData count together per object, why etcd imposes it, and name concrete alternatives.

for a senior

Lead with the control-plane cost model (etcd write amplification, kubelet caching, watch fan-out) and choose an alternative based on change cadence and startup-dependency risk.

for a principal

Set organisation-level policy: size and churn budgets for control-plane objects, where large artefacts live, and how config delivery stays reviewable and revertible when it moves outside the API server.

## Where the number comes from Every Kubernetes API object is serialised and stored as a single value in etcd. etcd's default `--max-request-bytes` is 1.5 MiB, and it warns above roughly that; going higher is explicitly discouraged by the etcd maintainers because large values inflate memory use, slow down Raft replication, and lengthen compaction. Kubernetes therefore validates ConfigMap (and Secret) content against a limit of 1 MiB (1048576 bytes) across `data` plus `binaryData`, rejecting the object at admission with a clear message. The limit applies to the object, not per key, and base64 expansion of `binaryData` counts. This is not a knob for an application team. Raising etcd's request size affects the whole cluster and is the kind of change that makes control-plane incidents worse, so "I'd increase the etcd limit" is a poor interview answer. ## The costs that make the limit sensible Size is only part of the story. Consider what a large ConfigMap does to a cluster: - **etcd write amplification.** Every update writes the entire object, replicates it to every etcd member, and adds a revision that must later be compacted and defragmented. A 900 KiB ConfigMap rewritten every minute is a meaningful sustained load. - **API-server memory and watch traffic.** The API server caches objects and fans changes out to every watcher. - **kubelet memory.** Each kubelet running a Pod that references the ConfigMap keeps it cached to serve volume mounts and env resolution. A large object referenced by a DaemonSet is multiplied by node count. - **Pod start latency.** Volume content must be materialised before the container starts. So the practical guidance is stricter than the hard limit: keep ConfigMaps in the kilobytes. If one approaches a megabyte, the design is usually wrong. ## Options when you exceed it **1. Split across several ConfigMaps.** If the payload is naturally many files (a rules directory, a set of dashboards), create several ConfigMaps and mount them into the same parent directory at different subpaths, or use a `projected` volume to merge several sources into one directory tree. This keeps each object small and lets you update one part without rewriting the whole. It does not help for one indivisible large file. **2. Bake it into the image.** If the data changes on the same cadence as the code, it belongs in the image: registries are built for large layers, distribution is cached per node, and the content is immutable and versioned with the tag/digest. The classic example is a large static ruleset or a GeoIP database shipped with the service. **3. Mount a volume.** A PersistentVolume (or a shared, read-only ReadWriteMany/ReadOnlyMany volume) is the right home for datasets in the tens of megabytes and up. The Pod reads it as a filesystem, the control plane never sees the bytes. **4. Fetch at startup.** An init container downloads the payload from object storage (S3/GCS) or a configuration service into an `emptyDir` that the main container mounts. The ConfigMap then holds only the pointer — bucket, object key, version — which is exactly the kind of small, human-reviewable value it is designed for. The trade-off is a new startup dependency: if the bucket is unavailable, Pods do not start, so you need retries, timeouts and ideally a cached fallback baked into the image. **5. Sidecar with live pull.** A sidecar polls a config service and writes into a shared `emptyDir`, giving hot reload without ConfigMap size pressure. This adds an always-running component and its own failure modes; prefer it only when live reload is a genuine requirement. **6. Compress.** Storing a gzipped blob in `binaryData` and decompressing in an init container buys maybe an order of magnitude and is occasionally the pragmatic fix, but it destroys reviewability and diffability — you can no longer see a config change in a pull request. Treat it as a last resort. ## Related limits worth knowing The 1 MiB cap is not the only one you can hit. The total size of a Pod's environment block is bounded by the exec argument/environment limits of the kernel (`MAX_ARG_STRLEN`, typically 128 KiB per single variable), so a huge value injected as an env var can fail at exec time rather than at admission. The Pod object itself must also fit in etcd, so enormous inline env lists can push the Pod spec toward the same ceiling. And annotations on any object are limited to 256 KiB in total, which rules out the occasional idea of "just put it in an annotation". ## How to answer Give the etcd origin, state the number, then immediately reframe: the limit is a hint that ConfigMaps are for small, reviewable configuration. Name two or three alternatives with the trade-off attached — image baking couples config to release cadence, external fetch adds a startup dependency, PVs add storage operations. Avoid suggesting that the limit be raised.

  • Does the 1 MiB limit apply per key or per object, and does binaryData count?
    Per object: the API server sums the byte length of all `data` values and all `binaryData` values and rejects the object above 1048576 bytes. Because `binaryData` is base64-encoded on the wire, the encoded form is what you are budgeting against, so binary payloads reach the ceiling about a third sooner than their raw size suggests.
  • Why is raising etcd's max-request-bytes a bad answer here?
    It is a cluster-wide change that makes every large object cheaper to create and therefore more common, increasing etcd memory use, Raft replication time and compaction cost for everyone. etcd's own guidance discourages it, and the failure it causes — a slow or unstable control plane — is far worse than the inconvenience of splitting configuration.

saying these in an interview costs you the question

  • Proposing to raise the etcd request-size limit as the primary fix
  • Thinking the limit is per key rather than per object
  • Assuming annotations or labels are a workaround for large payloads
  • Ignoring that every kubelet with a consuming Pod caches the object in memory
  • Treating ConfigMaps as a general-purpose file store for datasets

context

resolving links…