When a downstream service (say, an email/notification service) goes completely offline for 30 minutes, how does the blast radius differ between a request-driven design where the upstream service calls it synchronously versus an event-driven design where the upstream service publishes an event for it?

## Where the failure gets absorbed The difference comes down to where the failure of the downstream service is absorbed: in a **synchronous design**, it's absorbed (or not) by the calling code at the moment of the call; in an **event-driven design**, it's absorbed by the broker's durability, deferred entirely out of the calling code's critical path. ## The synchronous case, concretely Walk through the synchronous case concretely. Say checkout, after successfully processing an order, makes a direct HTTP call to a notification service to send the confirmation email, and waits for that call to return before finishing its own response to the customer. When the notification service is offline, that HTTP call fails to connect or hangs until a timeout. Without protective measures, several bad things can happen at once: - Checkout's own request now takes as long as the timeout instead of returning instantly, degrading the customer's experience even though nothing about their order actually failed. - If checkout doesn't set a timeout at all, its own request thread or connection can hang indefinitely, and under load, enough hung threads/connections exhaust the pool, meaning checkout itself starts failing for unrelated customers whose orders have nothing to do with email. This is the textbook **cascading failure**: a genuinely low-priority downstream dependency ends up threatening a genuinely critical path purely because of how the two were wired together. ## The mitigations the synchronous world already has The standard synchronous-world mitigations exist precisely to contain this: - A short **timeout** bounds how long checkout waits. - A **circuit breaker** stops calling notification altogether once failures cross a threshold, failing fast instead of piling up hung calls. - And, often, teams simply decide that a synchronous call was the wrong choice for a non-critical side effect and should be moved off the critical path entirely — effectively **reinventing an event**. ## The event-driven case Now the event-driven case. Checkout, after its own transaction, publishes an `OrderConfirmationRequested` event to a broker and returns success to the customer without knowing or caring whether the notification service is up. When notification service is offline, nothing about checkout is affected at all — not latency, not thread/connection pools — because checkout's interaction with the broker (a local, fast, durable write) has already completed. The events for those 30 minutes simply accumulate in the topic/queue. When the notification service comes back online, its consumer resumes processing from wherever it left off and works through the backlog, and every customer eventually gets their confirmation email, just late. The **blast radius** of the outage is contained to "notifications are delayed," a much smaller and more tolerable failure than "checkout intermittently fails or slows down for unrelated customers." ## The risk the event-driven version still carries But the event-driven version still has real risk, not none. - **First, durability capacity.** The broker has to actually be able to buffer 30 minutes of backlog without hitting storage limits or retention expiry — if a topic's retention is shorter than the outage, events get dropped, silently converting "delayed" into "lost." - **Second, time-sensitive side effects** break the "eventually successful is fine" assumption: if the event in question is "send a password-reset link" or "send a payment-verification code" rather than "send a receipt," a 30-minute delay may render that message functionally useless, so the outage isn't harmless just because it's decoupled — it's a different kind of harm (a failed user journey) instead of a cascading one. - **Third, the backlog burst.** When the consumer recovers, it usually has to process a large backlog quickly, which can itself cause a secondary problem — a burst of many emails sent in the same minute can trip the email provider's own rate limits or look like spam behavior, so "catching up" isn't automatically graceful either. ## Why notification sending lives behind a queue This is close to the textbook justification for why most production systems put email/SMS/push-notification sending behind a queue rather than a direct synchronous call from the service that triggers it — checkout, signup, and password-reset flows all publish an event and let a dedicated, independently-scaled notification service consume it, precisely so that provider outages or rate-limit issues on the notification side never threaten the availability of core account or purchase flows. The trade they accept in return is exactly the caveat above: they must separately guard against time-sensitive tokens expiring by keeping those specific flows on a tighter, monitored SLA even though the transport is still async.