How do you design an automated check for a push notification when delivery to the receiving device is best-effort?

## Where your system ends A push notification is produced inside your system and then handed to a delivery service that is not yours. Past that handoff, everything that decides whether it appears is outside your control: whether the receiving device is awake and on a network, whether its system software groups two notifications into one, whether the registration the delivery service holds for that device is still valid, and how long the whole hop takes. There is no durable store you can re-read, so a check that saw no arrival cannot distinguish "not sent" from "not yet" from "never will". That is what **best-effort** means concretely for a case: absence is not evidence, and the waiting time has no honest upper bound you can derive. A single assertion of the shape *notify, then wait for it to appear* inherits all of that uncertainty and then reports it as a product failure. ## Split the check in two The design that survives is two checks with different jobs, different schedules and different consequences. | | Intent check | Delivery check | |---|---|---| | What it asserts | The product produced an outbound notification for the right recipient, with the right content, exactly once | Something addressed to the run's own receiver actually landed | | Where it observes | The last point inside your system — the outbound record or the handoff call | A receiver the run registered and controls | | How often | On every change | On a schedule | | A failure means | Your product is wrong | Something between you and the device may be wrong | | Gates the change | Yes | No | The intent check is the one that belongs in routine runs. It is fast, deterministic, and a failure there is unambiguously yours: the notification was not produced, went to the wrong recipient, carried the wrong content, or was produced twice. Nearly every defect a team actually ships in this area is visible right at that boundary. ## Standing up a receiver the run controls The delivery check needs a recipient the automation can inspect. That is a **push sink**: a registered receiver — a real device the team owns, an emulated environment, or a service that registers as a recipient on the run's behalf — which records what it is given and exposes it to the case. Three properties make one usable: - **It is registered by the run, not by a person**, so the case knows the exact recipient it is asserting against and can tell its own arrivals from anybody else's. - **It records rather than displays.** The case reads a stored payload, not a rendering. What a screen draws is a different and much weaker assertion. - **Its registration is re-established as part of setup.** A receiver registered once, months ago, quietly stops receiving, and a check built on it turns red for reasons that have nothing to do with the product. ## Bounding something unbounded You still have to decide how long the delivery check waits. There is no derivable answer, so choose a **settle window** you can justify from arrivals you have actually observed rather than from a number someone remembered, and treat exceeding it as *not observed* rather than *failed*. Then make the outcome informational: record whether the notification landed and how long it took, and let the trend across many runs be the signal. One missing arrival tells you nothing; an observed rate falling from most-of-the-time to rarely tells you a great deal. The failure mode to rule out by construction is the retry loop. Re-running the case until a notification eventually shows up converts an unreliable check into a slow unreliable check, and it destroys the trend data that was the only real signal in the first place. ## When the delivery check goes red Because this check spans systems you do not own, its triage order differs from an ordinary failure: 1. **Compare it with the intent check.** If the intent assertion is green, your product produced the notification, so the problem is downstream or in the receiver. 2. **Suspect the receiver before the product.** A stale registration, a throttled sink, or a device that dropped off the network explains most of these. 3. **Compare across receivers.** One receiver failing while another succeeds is a receiver problem; all of them failing at once is worth escalating. 4. **Only then look at content.** A payload the outbound record shows as correct but that the receiver stored differently is the genuinely interesting case, and the one worth waking somebody for. The point of the split is not to care less about delivery. It is that a change should be blocked only by evidence the team can act on, and the arrival of a push notification on a device on the far side of a service you do not run is not that evidence.