If the feature-flag evaluation service itself becomes unreachable or slow, what should a client SDK do, and how does the answer differ between a flag guarding a nice-to-have feature versus a flag acting as a kill switch for a risky dependency?

## How a production-grade SDK is architected Mechanistically, a production-grade flag SDK is architected so the actual "should the flag do X" check inside application code is always a cheap, local, synchronous operation -- a lookup into an in-memory (sometimes also on-disk) cache -- never a network round trip on the request's hot path. Underneath that cache, the SDK maintains its own connection to the flag service in the background: - **Polling** on an interval, such as every 30 seconds. - **Holding a persistent streaming connection** (Server-Sent Events or a WebSocket) that pushes updates as soon as someone changes a flag in the dashboard. When application code calls something like `flags.isEnabled` with a `defaultValue` parameter, the SDK returns whatever's in its local cache; if it has never successfully synced yet -- a true cold start, right after process boot before the first fetch completes -- it returns the literal `defaultValue` the caller hardcoded at the call site, because there's nothing else to return. ## Why the flag service must not become a single point of failure This architecture exists to prevent a subtle but serious problem: pulling a piece of release/ops config out of the binary and into an external service must not turn that external service into a new single point of failure for every request in the system. If evaluating a flag meant blocking a request on a synchronous call to the flag service, an outage or even elevated latency in that one dependency would degrade or take down every service that checks flags, which in a flag-heavy codebase is most of them. Decoupling evaluation from the network call, via cache-then-background-refresh, means the flag service can be fully down for minutes and application requests keep flowing using the last known-good state, completely unaffected in the hot path. ## The trade-off: staleness That design choice creates a real trade-off: staleness. Between the moment someone flips a flag in the dashboard and the moment every running instance's local cache reflects that change, there's a window: | Sync mode | Staleness window | |---|---| | Poll-based SDKs | The polling interval | | Streaming SDKs | Typically sub-second | The window is never provably instant across a fleet of instances that might be mid-restart or behind a flaky network segment. For an ordinary release flag this staleness is harmless; for a kill switch being flipped during a live incident, that window is the response latency of your emergency lever, so teams running kill-switch flags at scale often specifically choose streaming SDKs and monitor SDK-to-service connectivity as its own health signal, because a kill switch that takes two minutes to propagate during an incident has failed at its one job. ## Failure modes The failure modes here are concrete and recur across real outages. 1. **Getting fail-open and fail-closed backwards** -- the first and most consequential. Every flag evaluation call typically carries an explicit default: the value returned when there's no cached state and no connectivity. For a normal, not-yet-launched feature, that default should be false or off, so any disruption to the flag service never accidentally exposes half-built functionality. But for a kill switch -- a flag whose on state means disable this risky subsystem -- the code's own default needs careful scrutiny: if the flag service goes down and the SDK falls back to a hardcoded off default meaning kill switch not engaged, that's fine if the switch was off anyway, but if the whole reason the flag service is unreachable is correlated with the same incident that would have required tripping the kill switch, the team is left with no working lever exactly when they need one. The safer design makes the guarded, safe fallback code path require no live flag connectivity at all, rather than depending on the flag system to actively deliver an off signal during the very outage it's meant to protect against. 2. **Cold-start default drift**, a quiet, slow-burning failure: engineers hardcode a true default at a call site once a feature has been at 100% rollout for months and everyone's forgotten the flag is still being checked; then a genuine flag-service outage or a fresh instance boot before first sync causes that code to briefly serve the wrong, stale default instead of the last real production value, producing behavior nobody intended that's hard to reproduce because it only shows up during connectivity gaps. 3. **Third, a thundering-herd reconnect storm**, which self-hosted or gateway-fronted flag services can suffer -- hundreds of instances losing their streaming connection simultaneously, for example during a flag-service deploy, and all reconnecting or re-polling at once, which SDKs mitigate with jittered backoff. ## Where it shows up in the wild A concrete real-world instance of this design: LaunchDarkly's SDKs require callers to pass an explicit default value on every evaluation call, specifically documented as the value used when the SDK cannot reach the service or hasn't initialized yet, and the company additionally offers a Relay Proxy specifically so a local, low-latency, cacheable layer sits between application instances and the flag backend, reducing both blast radius and staleness during backend disruptions.