What does an admission engine's replicated copy of cluster objects give a counting rule, and how is it wrong?
answer
- possible, not accurate
- write commits, then the copy learns
- restart means empty and filling
- only the kinds someone nominated
- undercount reads as a pass
basics
~20 sIt supplies the siblings a request does not carry, by watching the API and holding a local copy of chosen kinds. That copy always reflects a slightly earlier moment, and it is empty and filling right after the engine restarts.
solid answer
~50 sThe engine subscribes to the kinds you nominate and keeps their objects in memory, so a rule counting LoadBalancer Services in a namespace has something to count. What it buys is possibility, not accuracy. A write is committed first and the change reaches the copy afterwards, so every decision reads a view of the recent past; a Service created milliseconds ago may not be in it. After a restart the copy starts empty and fills from an initial list, and a count taken during that window undercounts - which for a limit rule means a false pass, and for a fail-closed design means blocking every write until sync completes. Only the kinds you chose are present, and replicating a large, churning kind costs real memory. Net effect: the rule goes from impossible to usually right.
go deeper
Know that a policy engine can hold a local copy of some cluster objects so a rule can count them, and that this copy is separate from the request being decided.
Explain the ordering - write committed, change delivered, copy updated - and what an empty copy right after a restart does to a limit rule.
Be able to say which way your engine fails while unsynced, name the signals you monitor for lag, and justify the specific set of kinds you replicate.
Frame replication as buying a soft invariant at a real cost in memory and blast radius, and decide where a detective sweep must sit behind it.
## What the copy is for Counting and uniqueness rules need objects the admission request does not contain. The common answer is to have the policy engine hold a local, continuously updated copy of a chosen set of kinds - it lists them once, then watches for changes - so a rule can ask *how many LoadBalancer Services exist in this namespace* or *does any Ingress already claim this host* without a network call in the middle of somebody's apply. That is a genuine capability. It is also a specific and limited one, and an interviewer probing this leaf wants to hear the limits named rather than the feature described. ## Limit 1: it is a read of the past The ordering is fixed and you cannot change it: the API server persists a write, then the change is delivered to watchers, then the local copy is updated. Every one of those steps takes time. A decision made in that gap is made against a state that does not include the write, even though the write already happened. For a limit of three, that means the fourth Service can be admitted if it arrives while the third is still propagating. The rule is not buggy - it answered correctly for the state it had. The state was simply behind. Say it precisely in an interview: **the copy is never the current cluster; it is the cluster as of a moment ago.** How far behind depends on watch delivery and the engine's own processing, and it stretches under load - exactly when writes are frequent and conflicts most likely. ## Limit 2: cold start When the engine process restarts - a rollout, an eviction, a node failure - its copy begins empty and repopulates from an initial listing. Until that finishes, a namespace with ten Services can look like a namespace with none. You get to choose which way that fails, and choosing by accident is the mistake: - **Serve anyway.** Rules evaluate against a partial view. Counting rules undercount, so they admit things they should have refused; uniqueness rules see no conflict, so they allow duplicates. Silent false passes. - **Refuse to serve until synced.** The engine reports itself not ready, and with a fail-closed configuration every matching write in the cluster is rejected until the copy is warm. Loud, safe, and an outage if warm-up is slow on a large cluster. Neither is wrong in the abstract; discovering which one you have during an incident is. Know it in advance, and monitor both sync completion and replication lag as signals with an actual owner. ## Limit 3: only what you nominated The copy holds the kinds someone configured it to hold. A rule referring to a kind that was never replicated does not get an error that stops the world - it gets nothing, and a rule reading nothing generally decides nothing is wrong. That is a quiet false pass with a configuration change, not a code change, as its root cause. Whenever a rule starts depending on a new kind, replicating that kind is part of the same change. ## Limit 4: it is not free Replication means keeping objects in memory and processing every change to them. Nominating a narrow kind such as Ingresses is cheap. Nominating a hot, high-cardinality kind because it might be useful is how a policy engine becomes the most memory-hungry component in the cluster and a source of load on the API server. Replicate what your rules actually read. ## Limit 5: it cannot see a decision in flight The copy only ever holds objects that have been written. Two conflicting requests being decided at the same instant are, by definition, not yet objects, so neither decision can see the other - no amount of freshness fixes that, because the information does not exist yet in any store. ## How to talk about it The honest summary is a downgrade, not a defect: replication turns a rule that is impossible into a rule that is usually right. It reliably catches the sequential case - somebody creating a fourth LoadBalancer an hour later - and it cannot catch the simultaneous case or the window after a restart. If an invariant genuinely must hold, admission plus a replicated copy is one layer, and it needs a detective sweep behind it that finds the violations that slipped through.
- What should happen while the copy is still filling after a restart?Decide it deliberately rather than inherit it. Either the engine reports itself unready so writes are refused until sync completes - safe, but an outage if warm-up is slow - or it serves partial state and silently under-enforces. Pick one, document it, and alert on both sync completion and replication lag.
- Why not replicate every kind, just in case?Memory and churn. A large cluster's Pods, Events and Secrets are numerous and constantly changing, and holding them means the policy engine carries a copy of the cluster plus the load of every update. Replicate the narrow kinds your rules actually read, and add a kind in the same change as the rule that needs it.
It is a photograph of the room taken a second ago. Good enough to count the people who were standing there, useless for the one who just walked in.
saying these in an interview costs you the question
- Calls the replicated copy the current cluster state
- Assumes it is fully populated the instant the process starts
- Claims replication makes a counting rule exact
- Replicates every kind to be safe
- Never asks what happens when the copy lacks a kind