What must a team's own controller do to reconcile a new declared object type the way the platform's built-in loops do?

seniornice to knowfreq 26%

answer

same contract as the built-in loops
triggered by identity, not payload
read both states inside the pass
re-queue with backoff, never fail terminally
link created objects to their declarer

basics

~20 s

A custom controller must register a declared type, be triggered by an object's identity rather than an event payload, re-read declared and live state on every pass, act idempotently, and report progress instead of failing terminally.

solid answer

~40 s

The contract is the same one the built-in loops obey, and none of it is exotic. You register a new object type with a block users declare and a block the controller writes back. Your entry point receives only the identity of an object that may need attention; it re-reads the declared state and the live world itself, computes the difference, and drives toward the end state with actions that are safe to repeat. When it cannot finish — a dependency is missing, a downstream call failed — it records the reason on the object and asks to be re-queued with backoff rather than raising a terminal error. It links whatever it creates back to the declaring object so deletion propagates, and it expects to lose a write race occasionally and simply run again.

The contract is the same one the built-in loops obey, and none of it is exotic. You register a new object type with a block users declare and a block the controller writes back. Your entry point receives only the **identity** of an object that may need attention; it re-reads the declared state and the live world itself, computes the difference, and drives toward the end state with actions that are safe to repeat. When it cannot finish — a dependency is missing, a downstream call failed — it records the reason on the object and asks to be re-queued with backoff rather than raising a terminal error. It links whatever it creates back to the declaring object so deletion propagates, and it expects to lose a write race occasionally and simply run again.

code

pseudocode · 13 lines

pseudocode

reconcile(objectKey):
    declared = api.read(objectKey)
    if declared == none:
        return done                     # object gone; ownership links clean up what it made

    actual = readWorld(declared.selector)
    if actual matches declared:
        api.writeStatus(objectKey, ready = true, reason = "met")
        return done

    applyDifference(declared, actual)   # set-to-target, never "one more"
    api.writeStatus(objectKey, ready = false, reason = "converging")
    return retryAfter(30s)              # no terminal error, just another pass

go deeper

for a junior

Recall that the platform's own behaviour is not special: the same observe-diff-act contract is open to code your team writes against an object type you register.

for a middle

Explain why the entry point takes an object identity rather than the change: the pass must re-read current state, so an event payload would only be a stale hint.

for a senior

Show the failure handling you would write — record the reason on the object, re-queue with backoff, never block inside a pass, and re-run rather than force a write after a conflict.

for a principal

Weigh what you are taking on: a controller is a permanently running privileged writer with estate-wide blast radius, so the question is whether continuous behaviour justifies it over a pipeline step.

Why the contract is open at all

A cluster platform's built-in behaviours are not privileged magic. They are loops reading a declared state and acting on the difference, written against an API that stores and serves object types. Teams can add their own object type and their own loop on exactly the same terms, and the value is that a team-specific end state — this database has this many read copies, backed up on this cadence — becomes declarable, reviewable and self-correcting instead of a script somebody runs.

That only works if the new loop behaves like the others. A controller that behaves differently is a permanently running privileged writer that surprises everyone, which is a worse outcome than a script.

The contract, item by item

Two blocks per object. A block the user declares and a block the controller writes back describing what it observed and whether the end state is met. Users write the first; only the controller writes the second.
Triggered by identity, not payload. The entry point takes the identity of an object, not the change that caused the trigger. The trigger says look at this; the pass then reads current state. This is what makes the loop level-triggered, and what makes duplicate or missing triggers cost latency instead of correctness.
Read both sides inside the pass. The declared state and the live world are both re-read every time. Nothing is carried between passes, so a restart is uneventful.
Idempotent actions. Every action is phrased as a target rather than an increment, and anything created carries a deterministic identity so a repeat is a no-op.
No terminal error for an unmet end state. When it cannot finish, the controller records why on the object and asks to be retried later with backoff. Failing the object outright takes away the platform's ability to recover when the blocker clears.
Never block inside a pass. Waiting inside the pass for something external holds a worker and stalls every other object. Return and be re-queued instead.
Ownership links on everything it creates. Created objects point back to the declaring object, so deleting the declaration cleans them up and a later pass recognises its own earlier work.
Tolerate concurrent writers. Writes can be rejected because someone else updated the object first. The correct response is to run again against fresher state, not to force the write.

What the status block buys

Reader	What it uses the written-back block for
A human during an incident	whether the end state is met, and the reason it is not
Another controller	the input to its own pass, which is how loops compose without calling each other
Automation that submitted the declaration	the signal that the intent is actually satisfied, rather than merely recorded

This composition property is the quiet reason the contract matters. Controllers that only read declared state and write status never need to know about one another; a chain of them converges because each one's output is another's input.

The failure modes worth naming

Two writers on one field. If two controllers both drive the same field toward different targets, each pass reverts the other and the object flaps forever. The fix is ownership: exactly one writer per object or per field, decided deliberately.
Hidden memory. A controller that caches decisions between passes works beautifully until it restarts, at which point its behaviour changes in ways nobody can reproduce.
Terminal failures. Marking an object failed because a dependency was missing converts a self-healing situation into a ticket.
Unbounded blast radius. A loop with permission to write across the whole estate, acting on every object at once, will apply a bug everywhere in one pass. Scoping and rate-limiting a controller is part of building one.

Deciding whether to build one

The honest trade-off: a controller is code that runs forever with write access, and it must be on-called, upgraded and reasoned about during incidents. That earns its place when the behaviour genuinely needs to be continuous — something that must be re-established whenever it drifts, for objects created by people who will never run your tooling. When the behaviour is a one-off transformation at deploy time, a pipeline step is smaller, simpler and easier to reason about, and choosing it is not a failure of ambition.

## Why the contract is open at all A cluster platform's built-in behaviours are not privileged magic. They are loops reading a declared state and acting on the difference, written against an API that stores and serves object types. Teams can add their own object type and their own loop on exactly the same terms, and the value is that a team-specific end state — *this database has this many read copies, backed up on this cadence* — becomes declarable, reviewable and self-correcting instead of a script somebody runs. That only works if the new loop behaves like the others. A controller that behaves differently is a permanently running privileged writer that surprises everyone, which is a worse outcome than a script. ## The contract, item by item 1. **Two blocks per object.** A block the user declares and a block the controller writes back describing what it observed and whether the end state is met. Users write the first; only the controller writes the second. 2. **Triggered by identity, not payload.** The entry point takes the identity of an object, not the change that caused the trigger. The trigger says *look at this*; the pass then reads current state. This is what makes the loop level-triggered, and what makes duplicate or missing triggers cost latency instead of correctness. 3. **Read both sides inside the pass.** The declared state and the live world are both re-read every time. Nothing is carried between passes, so a restart is uneventful. 4. **Idempotent actions.** Every action is phrased as a target rather than an increment, and anything created carries a deterministic identity so a repeat is a no-op. 5. **No terminal error for an unmet end state.** When it cannot finish, the controller records why on the object and asks to be retried later with backoff. Failing the object outright takes away the platform's ability to recover when the blocker clears. 6. **Never block inside a pass.** Waiting inside the pass for something external holds a worker and stalls every other object. Return and be re-queued instead. 7. **Ownership links on everything it creates.** Created objects point back to the declaring object, so deleting the declaration cleans them up and a later pass recognises its own earlier work. 8. **Tolerate concurrent writers.** Writes can be rejected because someone else updated the object first. The correct response is to run again against fresher state, not to force the write. ## What the status block buys | Reader | What it uses the written-back block for | |---|---| | A human during an incident | whether the end state is met, and the reason it is not | | Another controller | the input to its own pass, which is how loops compose without calling each other | | Automation that submitted the declaration | the signal that the intent is actually satisfied, rather than merely recorded | This composition property is the quiet reason the contract matters. Controllers that only read declared state and write status never need to know about one another; a chain of them converges because each one's output is another's input. ## The failure modes worth naming - **Two writers on one field.** If two controllers both drive the same field toward different targets, each pass reverts the other and the object flaps forever. The fix is ownership: exactly one writer per object or per field, decided deliberately. - **Hidden memory.** A controller that caches decisions between passes works beautifully until it restarts, at which point its behaviour changes in ways nobody can reproduce. - **Terminal failures.** Marking an object failed because a dependency was missing converts a self-healing situation into a ticket. - **Unbounded blast radius.** A loop with permission to write across the whole estate, acting on every object at once, will apply a bug everywhere in one pass. Scoping and rate-limiting a controller is part of building one. ## Deciding whether to build one The honest trade-off: a controller is code that runs forever with write access, and it must be on-called, upgraded and reasoned about during incidents. That earns its place when the behaviour genuinely needs to be **continuous** — something that must be re-established whenever it drifts, for objects created by people who will never run your tooling. When the behaviour is a one-off transformation at deploy time, a pipeline step is smaller, simpler and easier to reason about, and choosing it is not a failure of ambition.

What does the controller write back, and who reads it?
A status block: what it observed, whether the end state is met, and why not. Humans read it during incidents, automation reads it to know the intent is satisfied rather than merely recorded, and other controllers read it as the input to their own passes. That last one is how loops compose without ever calling each other directly.
What happens if two controllers try to drive the same object?
They fight. Each pass reverts the other's work and the object flaps indefinitely, burning write capacity and producing an event trail that looks like instability in the workload rather than in the automation. The fix is ownership decided up front: exactly one writer per object, or per field where the platform tracks writers that finely.
When is a controller the wrong answer?
When the behaviour is a one-off transformation rather than something that must be continuously re-established. A controller is a permanently running privileged writer that has to be on-called and upgraded; if a deploy-time pipeline step achieves the same end state, it carries far less operational weight and a much smaller blast radius.

saying these in an interview costs you the question

Feeds the change event's contents into the reconcile logic as its input
Keeps decisions between passes and resumes instead of re-deriving state
Returns a terminal error when a dependency is simply not ready yet
Blocks inside a pass waiting for something external to appear
Leaves created objects unlinked, so nothing cleans them up on deletion
Assumes it is the only writer and forces its write after a conflict

What must a team's own controller do to reconcile a new declared object type the way the platform's built-in loops do?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Why the contract is open at all

The contract, item by item

What the status block buys

The failure modes worth naming

Deciding whether to build one

▾common follow-ups3 questions

▾red flags6 flags

Why the contract is open at all

The contract, item by item

What the status block buys

The failure modes worth naming

Deciding whether to build one

answer

go deeper