skip to content

controller-runtime & Kubebuilder

controller-runtime hands you a manager, an informer-backed cached client and a Reconcile(ctx, req) to fill in, while Kubebuilder and Operator SDK scaffold the CRD, RBAC markers and an envtest harness. It checks you have really written a controller.

part ofKubernetesread it →
on this pageshow

questions

4

In a Kubebuilder project built on controller-runtime, what does the framework provide, and what do you actually write inside Reconcile(ctx, req)?

level: juniorimportance: must knowfreq 58%

answer

  1. library versus scaffolding
  2. one manager, shared caches
  3. req is only namespace/name
  4. Get, converge children, write status
  5. markers become ClusterRole and CRD

basics

~20 s

controller-runtime supplies a manager that runs shared caches, clients, work queues, leader election and metrics; Kubebuilder scaffolds the API types, markers and Makefile. You write Reconcile: fetch the object named by req, converge actual state to spec, then update status.

solid answer

~40 s

controller-runtime gives you a `Manager` (from `ctrl.NewManager`) that owns a shared informer cache, a client, the scheme, leader election, health probes and a metrics endpoint, and it starts every controller registered on it. You register a controller with the builder, `ctrl.NewControllerManagedBy(mgr).For(&GameSession{}).Owns(&appsv1.Deployment{}).Complete(r)`, and implement one method: `Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)`. The `req` carries only a namespace and a name. Inside, you `Get` the object, stop quietly if it is gone (`client.IgnoreNotFound`), compute the child objects it should have, create or patch them, and write what you observed to the `/status` subresource. Kubebuilder scaffolds the Go types, `SetupWithManager`, `cmd/main.go`, the `config/` Kustomize tree and a Makefile. `// +kubebuilder:rbac` and `// +kubebuilder:validation` markers become the ClusterRole and the CRD schema when you run `make manifests`.

code

go · 7 lines
go
func (r *GameSessionReconciler) SetupWithManager(mgr ctrl.Manager) error {
	return ctrl.NewControllerManagedBy(mgr).
		For(&gamesv1.GameSession{}).
		Owns(&appsv1.Deployment{}).
		Owns(&corev1.Service{}).
		Complete(r)
}

go deeper

for a junior

Recall the split: controller-runtime is the library with the manager and client, and Kubebuilder is the generator. Be able to write the signature Reconcile(ctx, req) and name the get, converge, update-status steps.

for a middle

Explain what the manager starts and in what order: caches sync before controllers run. Show how For and Owns route events to one reconciler, and how markers become YAML through make manifests.

for a senior

Show you have shipped one: leader election for multiple replicas, status written through the subresource, RBAC kept minimal by pruning markers, and SetupWithManager kept declarative rather than scattered through main.

for a principal

Weigh the framework choice for a platform: Go with controller-runtime versus the Java Operator SDK for a JVM team, the cost of owning generated scaffolding, and how framework upgrades are tracked across many operators.

## Two layers: library and scaffolding A **controller** is a program that watches Kubernetes objects and keeps making the cluster match what those objects declare. Writing one from scratch with raw client-go means wiring up informers, listers, work queues, rate limiters and leader election by hand. Two projects remove most of that work: - **controller-runtime** is a Go library (`sigs.k8s.io/controller-runtime`). It provides the runtime pieces: a manager, a cache-backed client, a controller builder, event handlers and predicates. - **Kubebuilder** is a CLI that generates a project laid out around controller-runtime, plus the Makefile targets that produce manifests from code. **Operator SDK** reuses the Kubebuilder layout for Go projects and adds packaging on top. ## What the manager gives you `ctrl.NewManager(cfg, ctrl.Options{...})` returns a **Manager**. Everything else hangs off it: - `mgr.GetClient()` returns a **client** that reads from a shared informer cache and writes straight to the API server. - `mgr.GetCache()` returns that shared cache. Two controllers watching Deployments share one informer. - `mgr.GetScheme()` returns the type registry that maps Go structs to group/version/kind. - **Leader election** is controlled by `LeaderElection: true` in the options (the scaffold wires it to a `--leader-elect` flag) and uses a `Lease` object by default. With two replicas of the operator, only the leader runs the controllers. - A **metrics** endpoint and **health/readiness probes** are also served. - `mgr.Start(ctx)` starts the caches, waits for them to sync, starts every controller and blocks until the context is cancelled. ## What Kubebuilder scaffolds | Path | Purpose | |---|---| | `api/v1/gamesession_types.go` | Spec and status structs, with markers | | `internal/controller/gamesession_controller.go` | The reconciler struct, `Reconcile` and `SetupWithManager` | | `cmd/main.go` | Builds the manager, registers controllers, starts it | | `config/crd`, `config/rbac`, `config/manager` | Kustomize bases for the generated YAML | | `internal/controller/suite_test.go` | An envtest-based test harness | | `Makefile` | `manifests`, `generate`, `test`, `run`, `deploy` targets | **Markers** are Go comments that `controller-gen` reads. `// +kubebuilder:rbac:groups=games.example.com,resources=gamesessions,verbs=get;list;watch` becomes a rule in the generated ClusterRole (`manager-role`). `// +kubebuilder:subresource:status` and `// +kubebuilder:validation:Minimum=1` shape the CRD. `make manifests` runs `controller-gen rbac:roleName=manager-role crd webhook ... output:crd:artifacts:config=config/crd/bases`. `make generate` produces the `DeepCopy` methods every API type needs. ## Writing Reconcile The signature is `Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error)`. `req` holds only `NamespacedName`: no event type and no old or new object. A typical body for a multiplayer game-session backend: 1. **Fetch** the `GameSession` named by `req`. If it is not found, return `ctrl.Result{}, client.IgnoreNotFound(err)`. 2. **Compute** the desired children, such as a Deployment of match servers and a Service. 3. **Converge** each child with `controllerutil.CreateOrUpdate`, setting the owner with `controllerutil.SetControllerReference` so that `Owns()` routes the child's events back here. 4. **Report** what you observed through `r.Status().Update` or `Patch`. 5. **Return** `ctrl.Result{}` when done, an error to be retried, or `ctrl.Result{RequeueAfter: d}` to be called again later. ```go // +kubebuilder:rbac:groups=games.example.com,resources=gamesessions,verbs=get;list;watch // +kubebuilder:rbac:groups=games.example.com,resources=gamesessions/status,verbs=get;update;patch // +kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete func (r *GameSessionReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { var gs gamesv1.GameSession if err := r.Get(ctx, req.NamespacedName, &gs); err != nil { return ctrl.Result{}, client.IgnoreNotFound(err) } dep := &appsv1.Deployment{ObjectMeta: metav1.ObjectMeta{Name: gs.Name + "-servers", Namespace: gs.Namespace}} if _, err := controllerutil.CreateOrUpdate(ctx, r.Client, dep, func() error { mutateServers(dep, &gs) return controllerutil.SetControllerReference(&gs, dep, r.Scheme) }); err != nil { return ctrl.Result{}, err } gs.Status.ReadyServers = dep.Status.ReadyReplicas return ctrl.Result{}, r.Status().Update(ctx, &gs) } ``` Idempotency, backoff and finalizers are the reconcile contract itself, and they belong to the reconcile-loop topic. This topic covers the framework that calls `Reconcile`. ## Common first-controller mistakes - **Building private clients or informers** inside the reconciler instead of using the manager's, which duplicates caches and bypasses leader election. - **Writing status with a plain `Update`** when the CRD enables the status subresource. The API server ignores status changes sent that way. - **Forgetting `SetControllerReference`**, so `Owns()` never routes the child's events back and drift on the Deployment goes unnoticed. - **Editing generated files** such as `zz_generated.deepcopy.go` or `config/rbac/role.yaml` by hand. The next `make generate` or `make manifests` overwrites them. - **Blocking in `Reconcile`** with sleeps while waiting for a child to become ready, instead of returning and letting the child's event or a `RequeueAfter` bring the reconciler back. ## The Java Operator SDK equivalent | controller-runtime | Java Operator SDK | |---|---| | `Reconcile(ctx, req)` | `UpdateControl<P> reconcile(P resource, Context<P> context)` | | `ctrl.Result{RequeueAfter: d}` | `UpdateControl...rescheduleAfter(d)` | | `r.Status().Update` | `UpdateControl.patchStatus(resource)` | | builder `For(...)` | `Reconciler<P>` plus `@ControllerConfiguration` | | finalizer handled by hand | implement `Cleaner<P>`; the SDK manages the finalizer | The Java SDK receives the **resource itself** rather than a key, but the model is the same: converge, then report status.

  • You added a `+kubebuilder:rbac` marker for Secrets but the operator logs forbidden errors in the cluster. What did you miss?
    Markers are only comments until `controller-gen` runs. `make manifests` regenerates `config/rbac/role.yaml` (the `manager-role` ClusterRole) and the CRDs under `config/crd/bases`, and the regenerated YAML then has to be deployed. Skip either step and the running ServiceAccount still has the old rules. The same applies to validation markers: the stale CRD in the cluster keeps the old schema.
  • How does the Java Operator SDK map onto this model?
    You implement `Reconciler<P>` and annotate the class with `@ControllerConfiguration`. `reconcile(P resource, Context<P> context)` returns an `UpdateControl` such as `patchStatus(resource)` or `noUpdate()`, optionally with `rescheduleAfter`. Implementing `Cleaner<P>` turns on automatic finalizer handling, and `cleanup` returns a `DeleteControl`. The SDK passes the resource itself instead of a namespace/name key, and it provides its own informer-backed event sources.
  • What happens when you run two replicas of a controller-runtime operator?
    With leader election enabled, both replicas start and compete for a `Lease`. Only the leader starts the controllers, so the other replica stays on standby, ready to take over. Runnables that do not need leader election, such as a webhook server, run on both replicas. If leader election is off, both replicas reconcile the same objects and fight each other.

controller-runtime is a building's plumbing and wiring, and Kubebuilder is the architect's standard floor plan. You still furnish one room, Reconcile, but the water and power already reach it.

  • Reconcile receives the changed object and the event type as arguments
  • Kubebuilder is a runtime library the operator links against instead of a scaffolding tool
  • RBAC markers grant permissions directly without regenerating and applying manifests
  • Each controller should build its own informers and clients rather than use the manager's
  • controller-runtime works only for custom resources, never for built-in kinds
open as a page

Why can controller-runtime's mgr.GetClient() return stale or unexpectedly expensive reads, and when should a controller use mgr.GetAPIReader() instead?

level: middleimportance: should knowfreq 46%

basics

~20 s

mgr.GetClient() serves Get and List from informer caches but sends writes to the API server, so a read can lag your own write, and the first read of a new kind starts a cluster-wide informer. mgr.GetAPIReader() reads live and caches nothing.

open as a page

A Kubernetes GameSession custom resource names a shared ConfigMap in spec.profileRef; with controller-runtime, how do you reconcile every referencing GameSession when that ConfigMap changes?

level: seniorimportance: should knowfreq 41%

basics

~20 s

Owns cannot help because the shared ConfigMap has no single controlling owner. Add Watches(&corev1.ConfigMap{}, handler.EnqueueRequestsFromMapFunc(fn)), where fn uses a field index on spec.profileRef to list referencing GameSessions and return their requests. Scope the ConfigMap cache with a selector.

open as a page

How does controller-runtime's envtest let you test a Kubernetes controller, and what does an envtest environment deliberately not run?

level: middleimportance: nice to knowfreq 33%

basics

~20 s

envtest starts real etcd and kube-apiserver binaries locally, installs your CRDs and returns a rest.Config for your manager. It runs no kube-controller-manager, scheduler or kubelet, so there is no garbage collection, no Pods running and no finished namespace deletion.

open as a page