In a multi-tenant platform where all tenants share one backend capacity pool, what techniques let you throttle for per-tenant fairness so a single heavy tenant can't degrade service for everyone else?

## What per-tenant fairness means Per-tenant fairness in throttling means the system tracks and bounds consumption at the level of the individual caller — a tenant, customer account, or API key — rather than, or in addition to, tracking it in aggregate across all callers. 1. The mechanism starts with **attribution**: every incoming request has to be tagged with an identity (a tenant ID from an auth token, an API key, a customer account) before the throttle can decide anything, because without that tag the system can only see total volume, not who is generating it. 2. Once requests are attributed, the throttle maintains a **separate consumption counter per tenant** against a separate budget per tenant, and admission decisions are made against that tenant's own budget rather than only against a single shared one. ## Why it matters The reason this matters is the **'noisy neighbor'** problem inherent to any shared resource pool. If a platform serves many tenants out of one backend with only a single global throttle, the global budget has no concept of who is using it — one tenant running a traffic-heavy campaign, retrying aggressively due to a bug, or simply being much larger than the others can consume the entire shared budget, and every other tenant's requests get rejected even though their own usage never changed. From the perspective of an unaffected tenant, this looks like an unexplained outage caused by someone else's behavior, which is a serious trust and SLA problem for a multi-tenant business — most platforms promise tenants some form of isolation, and throttling without per-tenant accounting silently breaks that promise under load. ## Allocation strategies and their trade-offs There are a few common allocation strategies, each with a real trade-off. - **A fixed quota per tenant** (e.g., every tenant gets the same N requests/second) is the simplest to implement and reason about, but it wastes capacity when usage is uneven — quiet tenants leave their share unused while active tenants are capped even though spare capacity exists elsewhere in the pool. - **A weighted or tiered allocation** ties each tenant's budget to their plan (a paying enterprise tenant gets a larger slice than a free-tier tenant), which better matches business intent but requires the throttle to know about billing tiers, coupling infrastructure logic to product/pricing logic. - **A dynamic fair-share scheme** reclaims unused budget from idle tenants and temporarily lends it to active ones — similar in spirit to how some job schedulers do fair-share CPU allocation — which improves overall utilization and is friendlier to legitimate bursty tenants, but is materially more complex to implement correctly and to reason about under load, since 'fair share right now' is a moving target that depends on what every other tenant is doing at the same moment. ## The operational cost The operational cost of per-tenant throttling is largely about **counters**: a single global counter can live in local memory on one node, but per-tenant counters at any real scale need to be visible across every node handling that tenant's traffic, which usually means a shared, low-latency store (an in-memory data store, a dedicated counting service) that itself has to be fast and available enough not to become the new bottleneck. This adds a network hop and a new dependency to the request path, and that dependency's own capacity and failure behavior now matters — a design that adds fairness at the cost of introducing a fragile shared component has arguably just moved the noisy-neighbor risk rather than removed it. ## Failure modes in production Failure modes to watch for in production: - **Without any per-tenant accounting at all**, a single tenant's traffic spike or retry bug causes platform-wide degradation that's hard to diagnose because the symptom (elevated rejection rate everywhere) doesn't point at the cause (one tenant's behavior) without per-tenant metrics. - **With naive fixed-equal quotas**, legitimate large tenants — often the platform's most valuable customers — are throttled even when the pool as a whole has spare capacity, which is a business problem disguised as a technical one. - **With dynamic fair-share schemes**, a bug in the reclaiming logic can let a burst of many simultaneously-active tenants collectively exceed real capacity, because the scheme assumed most tenants would stay quiet most of the time. ## Where it shows up A concrete real-world instance of this pattern is a platform like Stripe, which enforces per-account API rate limits so that one merchant's traffic surge — say, during their own flash sale — cannot degrade API latency or availability for every other merchant sharing the same platform, keeping each tenant's experience governed by their own usage rather than by their neighbors'.