A long-running queue consumer was declared under the run-to-completion contract and exits cleanly when the queue empties — what does the platform do?

## What the contract actually promised A run-to-completion workload is a promise about **outcomes**, not about **presence**. The declaration says: run this work until a declared number of runs have finished successfully, then stop. The platform's loop for it therefore asks "have enough runs succeeded yet?" — and when a process ends cleanly, the honest answer becomes yes. A consumer that drains a queue and returns when the queue is empty satisfies that literally. It ran. It ended without error. The completion count is met. The platform writes `finished: succeeded`, keeps the record, and has nothing left to do. There is no shortfall, because at no point did anyone declare that a copy should exist. ## Why this failure is quiet, and the service contract's is loud | | Declared as a job (what happened) | Declared as a service (what you wanted) | |---|---|---| | Invariant held | a count of successful runs | a count of live copies | | Clean exit | counts as success; workload finishes | a shortfall; another copy is created | | Unclean exit | a failed attempt against a retry budget | also a shortfall; another copy is created | | After the last exit | no copy exists, nothing is wrong | the platform keeps re-creating copies | | What you see | a green record and silence | repeated re-creation — obvious in the workload's own state | The asymmetry is the whole lesson. The wrong contract in the *other* direction — a batch declared as an always-on service — is noisy and self-announcing: the work runs again every time it finishes, and somebody notices by morning. The wrong contract in *this* direction is the dangerous one precisely because the platform has nothing to complain about. ## How you actually find it The platform is not the detector here; the work is. - **Backlog depth** on whatever the consumer reads climbs and does not come down. - **Downstream freshness** ages: records that should be minutes old are hours old. - **The workload's own record** shows a finished run with a suspiciously short duration and a completion time that matches when the trouble started. If your alerting is built only on "is the platform unhappy", this class of failure is invisible to it by construction. This is one of the clearest arguments for alerting on the *work* — queue age, output freshness, records processed per minute — rather than on the shape of what is running. ## The fix, and the case where the job contract was right If the consumer is meant to live forever, declare it as a replicated long-running service. Now the invariant is presence: any exit, clean or not, leaves the running count short and the platform creates another copy. A consumer that returns the instant its queue is empty will then be re-created continuously, which is not elegant — but it is *visible*, and you fix the process so its main loop waits for work instead of returning. There is a legitimate design where the job contract is the right one: a drain-this-batch-and-stop worker, started by a recurring schedule, deliberately processing whatever accumulated since the last run. The difference is not the code, it is the intent — you accept that nothing is consuming between runs, and you size the schedule against how stale the work is allowed to get. What makes the failure a failure is declaring the job contract for a workload whose requirement is continuous consumption. ## The direction that trips candidates Be explicit about which way the mistake runs, because both sentences are grammatical: 1. **Service declared as a job** — clean exit reads as success, nothing replaces it, the workload silently disappears. 2. **Job declared as a service** — clean exit reads as a missing copy, so the finished work is started again, and again. An interviewer asking this question is usually checking whether you can state the invariant each contract holds and then derive the symptom, rather than recalling an anecdote. The derivation is short: name the invariant, ask what a clean exit does to it, and the behaviour follows.