Your controller creates a Deployment, a Service and a Secret for each custom resource it manages. How do you ensure those objects are cleaned up when the custom resource is deleted, and how does Kubernetes garbage collection decide what to remove?

## Why ownerReferences exist Controllers create many objects on a user's behalf. Deleting them by hand in reconcile would be fragile: the controller might be down when the parent is deleted, and by the time it comes back the parent object is gone, so there is nothing left to tell it what to clean up. Kubernetes solves this in the API itself with **ownership metadata plus a cluster-wide garbage collector**. Each object may carry `metadata.ownerReferences`, a list of owners: ```yaml ownerReferences: - apiVersion: apps.example.com/v1 kind: App name: checkout uid: 0a5f... controller: true blockOwnerDeletion: true ``` The **UID** is the identity that matters. Names are reusable; UIDs are not. That is what prevents a newly created object with a recycled name from inheriting the old one's children. ## How the garbage collector works The garbage collector runs in kube-controller-manager. It maintains a graph of owners and dependents from watch data. When an owner is deleted (or is found never to have existed), every dependent that references that owner UID becomes eligible for deletion. This is asynchronous and cluster-wide; it does not require your controller to be running, which is precisely the point. **Propagation policies**, chosen by the deleter: - **Background** (default for most clients): the owner is removed straight away and the collector deletes dependents afterwards. Fast, but for a short window children exist without a parent. - **Foreground**: the API server adds a `foregroundDeletion` finalizer to the owner. The owner remains visible with `deletionTimestamp` set until all dependents that set `blockOwnerDeletion: true` are gone, then the owner is removed. Use it when something must not disappear from the API until its children are actually gone. - **Orphan**: the collector strips the ownerReference from dependents and leaves them running. Historically used for `kubectl delete --cascade=orphan`. `blockOwnerDeletion: true` on a dependent is what makes foreground deletion wait for it. Note that setting it requires the `delete` permission on the owner, which occasionally surprises controllers with tight RBAC. ## The controller field At most one ownerReference may have `controller: true`. It marks the *managing* controller and is used for adoption logic — a ReplicaSet will not adopt a Pod that is already controlled by someone else. For custom controllers, `SetControllerReference` sets it and errors if a different controller already owns the object, which is a useful guard against two controllers fighting over one child. ## Rules and gotchas 1. **Same namespace only.** A namespaced dependent must be owned by an owner in the same namespace. A cross-namespace reference is invalid; the collector treats the owner as non-existent and deletes the dependent. This is a classic source of "my ConfigMap keeps disappearing". 2. **Cluster-scoped owners only for cluster-scoped dependents.** A cluster-scoped object (a ClusterRole, a PV) cannot be owned by a namespaced custom resource. To clean those up you need a finalizer on the parent and explicit deletion in reconcile. 3. **UID mismatch is fatal to the child.** Delete and recreate the parent with the same name and the old children are garbage — they point at a UID that no longer exists. 4. **Nothing outside the cluster is covered.** Cloud buckets, DNS entries, database users: ownerReferences cannot help. That is what finalizers are for. 5. **Deletion is not instantaneous.** Background propagation means clients that immediately re-create the parent may briefly observe leftovers. Idempotent, deterministic-name reconciliation handles this. ## Ownership as a wiring mechanism, not just cleanup OwnerReferences also drive event routing. In controller-runtime, `Owns(&appsv1.Deployment{})` installs a handler that reads a Deployment's controller ownerReference and enqueues the *owner's* key. So when a managed Deployment's status changes, the parent reconciles and can update its own status. Without the reference, you would need a custom mapping function or an index. This dual role — cleanup and event mapping — is why setting ownership correctly is a habit rather than an option. ## Practical checklist for a reconciler - Call `SetControllerReference(cr, child, scheme)` before creating any child, and handle its error rather than ignoring it. - Never set ownership across namespaces or from a namespaced owner to a cluster-scoped object; delete those explicitly under a finalizer. - Use foreground deletion when ordering matters for the API surface; otherwise the default is fine. - Do not implement child cleanup by hand in reconcile when ownership suffices — the collector works even when your controller is offline. - Remember that ownership does not imply the same lifecycle for external resources. ## Interview framing Say ownerReferences plus the garbage collector, name the UID as the identity, list the three propagation modes, and then volunteer the two limits — same namespace / not cluster-scoped, and nothing outside the cluster. Mentioning that ownership also routes child events to the parent's reconcile shows you have wired a real controller.