How do leftover test identities produce both a false pass and a false failure in a messaging suite?

## One cause, two opposite symptoms Both lies come from the same fact: the identity carried state into a run that did not create it. Because a messaging case usually asserts on the *presence* of something that arrives from outside the product, leftover state can either supply that something when the product did not, or prevent it when the product did. The first gives you a false pass, the second a false failure, and the pair is why leftover identities corrupt a suite rather than merely inconveniencing it. | | False pass | False failure | |---|---|---| | What leftover state does | supplies the evidence the case looks for | blocks the delivery the case waits for | | Typical source | an earlier run's copy still in the mailbox; an already-confirmed account | a suppression entry, withdrawn consent, an expired number lease | | What the result says | the feature works | the feature is broken | | Reality | the feature may be entirely dead | the product behaved correctly | | How it is usually handled | never noticed | re-run until green, then forgotten | | Cost | releases ship on false confidence | investigation time, then eroded trust | ## The false pass, and why it is the dangerous one Consider the plainest possible assertion: *a message whose subject is "Welcome aboard" exists in this recipient's mailbox*. If that recipient was used a month ago and never expired, the mailbox already contains one. The assertion is satisfied the instant it runs. The product could have its entire sending path disabled and the case would still be green. The second form is quieter still. The case registers an address that is already registered and already confirmed from an earlier run. Depending on the product, one of two things happens: the registration is rejected and the case's error handling swallows it, or it is treated as a no-op and the confirmation the case then checks was set weeks ago. Either way the step under test never ran, and the case reports success on a state it inherited. What both have in common is that the case never established its **precondition**. It asserted a post-state without ever having demonstrated the pre-state, so nothing in it distinguishes "the product did this" from "this was already true". ## The false failure, and why it looks like flakiness The mirror image is an address the delivery side has an opinion about. An unsubscribe followed in an earlier case, a bounce from a mistyped variant, or a complaint recorded long ago writes a suppression entry, and a suppressed address never receives anything again — the send is dropped before it is attempted. Withdrawn consent produces the same observable outcome by a different route: the product correctly declines to send at all. A number whose lease has expired and been reassigned is a third: the code arrives somewhere, just not where the case is looking. These read as flakiness because they are selective. Fresh identities pass; the aged ones fail. The same code, the same product, different outcomes — which is exactly the profile that gets a case re-run rather than investigated. Re-running against a *different* identity makes it go green, confirming the wrong conclusion. ## Making a green result mean something The repairs are mechanical, and they cost very little: 1. **Expire identities on a schedule**, so no case inherits a mailbox with history. This removes the cause rather than detecting the symptom, and it fixes both directions at once. 2. **Assert the precondition first.** Check the mailbox is empty and the account does not exist *before* acting. A case that cannot state its starting condition cannot interpret its ending one. 3. **Assert a count, not an existence.** *Exactly one arrival* fails on a duplicate copy where *at least one* passes. 4. **Verify the case can fail.** Disable the sending path deliberately, once, and confirm the case turns red. A messaging case that stays green with the sender switched off is testing nothing, and this is the cheapest way to find that out. 5. **Fail loudly on suppression.** When the run can see the delivery side's status, treat a suppressed or unsubscribed recipient as an environment fault with its own message, not as a generic timeout. That converts a mysterious flake into a one-line diagnosis. ## The judgement to bring If asked to rank the two, rank the false pass first. A false failure is loud: it interrupts someone, it gets a look, and eventually somebody finds the suppression entry. A false pass is silent by construction — it produces exactly the signal everyone wants to see, it never asks for attention, and it can survive many releases while the notification path it claims to cover is dead. The moment a green messaging result stops being evidence, the suite has become a cost with no benefit, and identity hygiene is a large fraction of what keeps that from happening.