A scenario's Then only checks that the playlist screen shows the new order, and a silent reordering corruption survived nine nightly runs — how do you strengthen it?

### The Then is an oracle, and proxies make weak oracles An oracle is whatever decides that observed behaviour is wrong. In a scenario, the Then is it. The failure described here is not a missing assertion — there was an assertion, and it was green for nine consecutive nightly runs while roughly one reorder in 340 wrote a scrambled order to storage. The assertion was simply pointed at a proxy: the view the actor who made the change was looking at. That view was updated locally the moment the gesture completed, so it agreed with the intent regardless of what was stored. A green scenario, a broken promise, and a suite that produced false assurance for over a week. This is the specific reason "assert an observable outcome, not a screen state" is a rule rather than a stylistic preference. Screens are frequently *where* an outcome becomes visible, so the rule is not "never look at a screen". It is: name the promise in the Then, then check it where the promise actually lives, using the weakest coupling that still detects a violation. ### Move one — restate the promise Start with the text, before touching automation. "Then the playlist screen shows the new order" describes a display. "Then the playlist order starts with that track for every collaborator" describes what the product promised, and it immediately implies checks the first wording never suggested: someone other than the actor, and a state that outlives the gesture. Rewriting the sentence is what surfaces the missing checks; teams that go straight to the automation usually add a wait and call it fixed. ### Move two — observe from an independent vantage point The actor's own session is the least trustworthy place to confirm a persisted effect, because everything between the gesture and the pixels may be optimistic. Independence can be bought cheaply: re-read the playlist as a second collaborator, or re-request it in a way that cannot be served from the writer's local state. This is what would have caught the corruption on the first night — the scrambled order was in storage the whole time, visible to anyone who read it fresh. ### Move three — assert the invariant, not only the example Ordering rules have a companion invariant that is cheap to state and catches whole classes of corruption: the multiset of tracks is unchanged. A reorder that loses one track, duplicates another, or drops a track's attribution is a corruption the position assertion may not notice, because position one can be correct while position eleven is a duplicate. One extra assertion line — the playlist still holds the same 14 tracks, each exactly once — turns a narrow example into a check with real defect-detecting power. ### How far down should the assertion reach? There is a genuine tension. Reaching directly into storage from a step is the most sensitive check available and the most brittle: it couples business-facing prose to a schema, it breaks whenever storage changes for reasons no reader of the scenario cares about, and it lets a scenario pass while the read path that users depend on is broken. The usual resolution is to assert through the service's own public read path — the same route a collaborator's client would take — which is stable, observable and still sees the real stored state. Reserve direct inspection of internals for a narrow diagnostic check outside the business-facing suite, if at all. ### Silent versus loud failures The diagnostic question worth asking of any Then is: *what would this assertion look like if the behaviour were quietly wrong rather than loudly broken?* Loud failures — an error, a blank page, a rejected request — are caught by almost any assertion, including weak ones. Quiet failures are caught only by assertions aimed at the promise. A suite whose Thens are all proxies is tuned entirely for loud failures, which is why it can run for six hours a night, stay green, and still miss data being corrupted underneath it. ### What not to conclude Two overcorrections are common. The first is to bolt on many more assertions per scenario until each one covers everything, which reintroduces bundled Thens that stop at the first failure and scenarios nobody can read. Prefer one promise per line, and a separate scenario for a separate rule. The second is to conclude that the interface should never be observed. If a rule is about what a collaborator sees, seeing it is the outcome — the point is that the scenario must say so deliberately, rather than defaulting to the display because it was the easiest thing to check.