Run Shapes

Under its intended load a system has one job: serve everything, inside the response-time bound the product promised. Past that load the job changes, and the promise has to change with it. "It degrades gracefully" is the sentence teams reach for to describe the new promise, and it is the single most common reason an overload run produces no finding: it cannot be compared against anything. ## Why the phrase is not an assertion A test case asserts by comparing an observation against a value fixed in advance. "Gracefully" fixes nothing. Two engineers reading it picture different systems. One imagines a service that turns away a third of its traffic instantly and keeps the rest fast; the other imagines a service that slows down for everybody but eventually answers every request. Both are defensible designs, and a phrase rules out neither. So the run produces graphs and the judgement moves to whoever is in the room when the graphs appear. That is a demonstration rather than a run with a result, and nothing is recorded that a later change can be caught regressing against. The repair is mechanical. Before the run, split the offered work into outcome classes and put a bound on each class. ## The three clauses a testable rule needs **1. The accepted-work clause.** Name the share of offered requests the system is still expected to complete, and the response-time bound those completions must stay inside. Inside the intended limit this clause is trivially "all of them"; past the limit it becomes a floor, because turning some work away is now permitted. The floor is what makes the run falsifiable in the useful direction: a change that quietly cuts how much work survives overload breaks it. **2. The refusal clause.** The work that is not completed has to go somewhere, and "refused" is a legitimate destination. Bound it: how quickly a rejection must come back, and the requirement that the caller receives something it can recognise as a refusal rather than silence. The refusal bound is normally far tighter than the served bound, because a rejection that costs as much waiting as a success buys the caller nothing. **3. The never-events clause.** Some results fail the run whatever the timings say, because they are damage rather than slowness: work applied in part and left inconsistent, a request accepted and then lost with nobody told, a queue or in-flight set that grows without any ceiling. These are not traded against response times, and writing that down is what stops them being traded in the room afterwards. | Clause | What it fixes in advance | What a violation looks like | | --- | --- | --- | | Accepted work | a floor on the share completed, and their response-time bound | the floor is missed, or surviving requests exceed the bound | | Refusal | how fast a rejection returns, and that it is explicit | callers wait a full timeout before being told no | | Never-events | outcomes that are not tradeable against timing | half-applied work, silent loss, a backlog with no ceiling | ## The number everyone forgets: the pressure the rule holds at Behaviour past the limit is not one behaviour. A service may hold the accepted-work floor comfortably at twice its intended arrival rate and miss it badly at ten times that rate. A rule that does not say which pressure it applies to can be satisfied by almost any run, and two runs at different pressures get compared as though they were the same test. Write the offered rate into the rule, and accept that you are writing a small family of rules, one per pressure the product is actually worth defending. Deciding that nothing is promised above some pressure is a legitimate and often correct answer; leaving it unsaid is not. ## Writing one in ten minutes 1. State the intended limit you are exceeding, and this run's offered rate as a multiple of it. 2. Fix the accepted-work floor, and the response-time bound for the requests that survive. 3. Fix the refusal bound, and state that a refusal must be explicit to the caller. 4. List the never-events, and name the check that would detect each one. 5. Say which window the figures are read over, so a slow opening minute is not argued about later. 6. Get the owner of the user-facing promise to agree the rule before the run starts, not after. ## What the rule deliberately does not do It does not try to discover the pressure at which the system stops coping. That is a capacity question with its own method and its own answer, and folding it in makes one run answer two questions badly. It also does not prescribe how the system achieves the behaviour. The rule states outcomes; whether they are met by refusing early, by reducing what each response contains, or by queueing behind a ceiling is a design decision, and a rule that names the mechanism will fail the next perfectly good implementation for the wrong reason.