A Kyverno Pod rule blocks Deployments, but the same violation in a CronJob is admitted. Why?
answer
- a silent gap produces no evidence
- the applied YAML looked correct throughout
- coverage is configurable per controller kind
- read the derived rules back from the cluster
- check the autogen-controllers annotation
basics
~20 sMost likely the policy carries a pod-policies.kyverno.io/autogen-controllers annotation that narrows generation to a subset of kinds, so no CronJob variant was ever derived. Read the policy back from the cluster: if no autogen-cronjob- rule exists, CronJobs are outside the control.
solid answer
~40 sAutogen coverage is configurable, and a narrowed configuration fails silently. The `pod-policies.kyverno.io/autogen-controllers` annotation on the policy restricts which controller kinds get derived rules; the value `none` disables generation entirely. If someone set it to `Deployment,StatefulSet` - often long ago, to keep a policy small - then no `autogen-cronjob-<rule>` was derived and CronJobs never touch the control. Diagnose it by reading the policy back from the cluster and listing which derived rules exist, then compare that against the workload kinds actually running. The fix is to widen or remove the annotation and re-verify the derived set, not to hand-write a CronJob rule, which leaves the same gap open for Job and DaemonSet. Then check whether anything shipped through the gap while it was open.
go deeper
Know that Kyverno's coverage of workload kinds is configurable on the policy, so a rule can be genuinely absent for one kind without any error appearing.
Be able to name the annotation, explain what a narrowed value or none does to the derived rules, and describe how you would confirm which kinds are covered.
Show the full diagnosis: derived set first, annotation second, rule logic third, match and exclude fourth - and then ask what shipped through the gap while it was open.
Own the review standard that closes this class of gap - the derived rule set is the artefact reviewed, and any narrowing must be argued for and named in the change that introduces it.
## The shape of the failure This is the most instructive failure mode in Kyverno's autogen, because **nothing goes wrong**. No error, no warning, no failed request. A control that everyone believes is universal is quietly absent for one workload kind, and the only evidence is the absence of a rejection that should have happened. Take the control: `no container may receive a Secret through env.valueFrom.secretKeyRef or envFrom.secretRef; mount it as a projected volume instead`. Deployments carrying `secretKeyRef` are refused all day. A nightly CronJob doing exactly the same thing is admitted, has been for months, and nobody noticed because admitted requests produce no artefact to notice. ## The usual cause The policy carries an annotation: ``` pod-policies.kyverno.io/autogen-controllers: Deployment,StatefulSet ``` That annotation narrows which controller kinds Kyverno derives rules for. With that value, `autogen-<rule>` covers Deployment and StatefulSet, and no `autogen-cronjob-<rule>` is derived at all. Set the annotation to `none` and no derived rules exist at all - only directly submitted Pods are covered, which in most clusters is close to nothing. These narrowings are rarely malicious and rarely recent. They get added when a policy is first written - copied from an example, or added to keep the derived set small - and then survive every later review because the rule *logic* looks complete and correct. The annotation is metadata; reviewers read the rule. ## Diagnosis, in order 1. **Read the policy back from the cluster, not from the repository.** The repository holds what you authored; the cluster holds the derived set that is actually enforced. List which derived rules exist. If there is no rule prefixed `autogen-cronjob-`, you have your answer immediately. 2. **Read the annotation on the policy.** A narrowed list or `none` explains a missing variant outright. 3. **If the CronJob variant does exist**, the gap is not coverage but logic: the rule body does not survive relocation to the deeper `spec.jobTemplate.spec.template.spec` path - usually because a path prefix was hand-written into a rule that also matches Pod. 4. **Check the match and exclude blocks** for a namespace or label restriction that happens to exclude where the CronJobs run. That is a different gap with the same symptom, and it is worth eliminating before you blame autogen. ## Fixing it properly The tempting fix is to hand-write a CronJob rule. Resist it. It closes exactly one hole, leaves Job and DaemonSet open if they were also excluded, and adds a second rule to keep in sync with the first - so the next person changing the control changes one of them. The correct fix is to widen or delete the annotation, apply, and then **verify by reading the derived set back**. Verification is not optional here: the whole point of this failure is that the authored YAML looked right the entire time. ## The part that is not technical Once the gap is closed, two questions follow, and a senior engineer is expected to raise them without being asked. **What went through while it was open?** The rule was not enforcing for that kind, so no record of a decision exists for those workloads. You have to establish current state directly - which CronJobs are injecting Secrets as environment variables today - and treat that as the remediation backlog. "The policy was in place" is not an answer, because for that kind it was not. **How does the same gap get caught next time?** The durable answer is that the derived rule set, not the authored rule, is what gets reviewed - and that any narrowing annotation has to be argued for in the change that introduces it, with the excluded kinds named. A narrowing that exists for a reason and says so is fine. A narrowing nobody can explain is a hole with an explanation attached. ## Why this generalises Every gate has a configuration surface separate from its logic - what it matches, where it applies, which kinds it was derived for. Reviewing the logic while ignoring the surface is how controls end up covering less than everybody believes. Autogen just makes the lesson unusually crisp, because the missing coverage is a single word in an annotation and the symptom is nothing at all.
- Why is hand-writing a CronJob rule the wrong fix?It closes one hole and leaves the others - Job and DaemonSet may be excluded by the same annotation - and it creates a second copy of the control that will drift from the first. Widen the annotation instead, then verify the derived set.
- The derived CronJob rule does exist, but CronJobs still pass. What now?Then coverage is fine and the logic is not. Check whether the authored rule hard-codes a path prefix, which autogen then prefixes again, and check the match and exclude blocks for a namespace or label restriction that happens to skip where the CronJobs run.
- What do you say when asked whether the control was in place for the last two quarters?For Deployments, yes; for CronJobs, no - and be precise about that split rather than averaging it. Then establish current state directly for the uncovered kind, because an admitted request left no record and the absence of denials proves nothing.
- Is narrowing autogen ever legitimate?Yes, when a rule is genuinely meaningful only for some kinds. The requirement is that the narrowing be deliberate, named in the change that adds it, and reviewed alongside the rule logic rather than passed over as metadata.
saying these in an interview costs you the question
- Assumes autogen coverage is unconditional and cannot be narrowed
- Blames the CronJob schedule or its Job timing
- Adds a hand-written CronJob rule and leaves the annotation alone
- Treats an absence of denials as evidence the control was working
- Reviews the authored rule and never the derived set