How does a platform controller provision Kubernetes namespaces on request, from a developer's custom resource to a ready namespace?

## The problem self-service provisioning solves A `Namespace` is a **cluster-scoped** object. Letting developers create namespaces directly means granting them a cluster-wide permission, and a namespace created by hand arrives *empty*: no access bindings for the team, none of the platform's guardrails, no ownership labels. Filing a ticket with the platform team fixes that but turns a thirty-second need into a day's wait. **Self-service provisioning** gives developers a narrow request object instead, and puts a **controller** in charge of turning each request into a fully configured namespace. ## The moving parts - **The request resource.** A CustomResourceDefinition, owned by the platform team, for a kind such as `TeamSpace`. It carries only what a developer should choose: team name, owning group, environment tier, cost centre. - **The controller.** A process — typically written with controller-runtime or Kubebuilder — that watches `TeamSpace` objects and the objects it creates. It runs under its own ServiceAccount, which needs broad rights; that makes it a privileged platform component to protect and review. - **The produced objects.** The `Namespace` itself plus the platform's standard bundle: team access, and the guardrails the tenancy model prescribes. Which guardrails belong in that bundle is a tenancy-model decision, not a provisioning one. ## The reconcile loop, step by step 1. A developer commits a `TeamSpace` named `fraud-rules` (or submits it through a portal). The API server stores it. 2. The controller's watch fires, and `Reconcile` reads the desired state from the request. 3. It creates or updates the `Namespace` `fraud-rules`, adding platform labels such as `app.kubernetes.io/managed-by`. Kubernetes adds `kubernetes.io/metadata.name` on its own. 4. It sets an **`ownerReferences`** entry on the Namespace pointing at the `TeamSpace`. The garbage collector allows this only because the owner is also cluster-scoped; a cluster-scoped object that names a namespaced owner is rejected as invalid by the garbage collector. 5. It creates a `RoleBinding` inside the namespace binding the team's group to a built-in ClusterRole such as `edit`, then the rest of the bundle. 6. It writes `status.conditions` — for example `Ready=True` — so the developer and any tooling can tell when the namespace is usable. The loop is **level-triggered and idempotent**: every run compares desired with actual and fixes the difference. If someone deletes the team's RoleBinding, the next reconcile recreates it. If the platform changes the bundle, every existing namespace converges on the next resync. ## Design choices that matter | Choice | Option A | Option B | |---|---|---| | Request scope | Cluster-scoped request (can own the Namespace) | Namespaced request in an intake namespace (cannot be an owner; needs a finalizer and labels to link) | | Request channel | Git, reviewed and reconciled by a GitOps controller | Portal or CLI writing directly to the API | | Approval | Automatic for low tiers | Human approval for production tiers | | Deletion | Cascading delete of the namespace | Finalizer that blocks or requires confirmation | ## The dangerous edge: deletion Because the Namespace is owned by the request, **deleting the request deletes the namespace**, and deleting a namespace deletes everything inside it — Deployments, Secrets, PersistentVolumeClaims. A typo in a Git directory or a careless cleanup script can wipe a team's environment. Common mitigations: - a **finalizer** on the request that refuses to proceed unless the namespace is empty or an explicit annotation confirms intent - leaving production namespaces *without* an owner reference, so the controller manages them but the garbage collector never cascades - admission policy that blocks deleting requests for production tiers ## Why not just a script? A one-shot script creates the namespace once and never looks again. A controller keeps looking: it repairs drift, rolls bundle changes out to existing namespaces, and reports readiness through status. That continuous reconciliation — not the creation itself — is what makes provisioning *self-service* rather than *self-served tickets*. ## Checking the result A developer confirms the outcome with ordinary commands: `kubectl get namespace fraud-rules --show-labels`, `kubectl get rolebindings -n fraud-rules`, and `kubectl auth can-i create deployments -n fraud-rules` to prove the access binding took effect.