You operate a shared service where many tenants contend for one bounded pool of workers. How do you decide how much fairness to enforce between them, and what would you measure to know the choice was right?

## Start from the promise, not the primitive Fairness is not a virtue to maximise; it is a bound you buy at a price. The first move is to write down what each class of work is owed when the system is saturated. Typical contracts: - *Best-effort*: no guarantee, work may wait arbitrarily. Cheapest, and perfectly fine for internal batch jobs. - *Minimum share*: tenant T receives at least X% of capacity whenever it has demand. This is the usual multi-tenant promise. - *Bounded wait*: no request waits more than D. Strongest and most expensive, because it constrains the tail directly. Only after that sentence exists can you evaluate mechanisms, because each mechanism buys exactly one of those bounds. ## Isolation beats ordering The cheapest way to prevent one tenant starving another is not to order the shared queue more carefully — it is to stop them sharing the contended resource without limit. Concretely: - **Per-tenant concurrency caps**: tenant T may hold at most k workers. A runaway tenant now consumes k, not the pool. Cost: a small counter per tenant; no handoff cost per operation. - **Separate queues or pools per class**: latency-critical and batch work never share a queue. Cost: capacity fragmentation, since an idle pool cannot help a busy one — mitigate with borrowing plus a hard reservation. - **Weighted fair queueing / virtual time**: one shared pool, but each tenant has a weight and a virtual clock; the scheduler always serves the tenant whose virtual finish time is earliest. Every tenant with demand progresses, in proportion to its weight. Cost: bookkeeping per dequeue, and a real need for accurate cost accounting. Strict global FIFO — the most obviously "fair" option — is usually the wrong pick, because it protects nothing except arrival order, and arrival order is exactly what a spamming tenant controls. A tenant submitting a million items still gets a million turns. ## Cost asymmetry is where fairness schemes break Counting requests is fair only when requests cost the same. In practice one tenant's request may be a hundred times heavier. Two disciplines fix this: charge by **measured service time** (update the virtual clock by actual work done, as stride and virtual-time schedulers do), or **normalise units** (cost-weighted tokens, e.g. estimated rows scanned) and reconcile against measured cost afterwards. Without this, a nominally fair policy systematically transfers capacity to the expensive tenant, and your dashboards will show equal request shares while latency for everyone else degrades. ## Admission control removes the regime Starvation and unbounded wait exist only under sustained saturation. Bounded queues, per-tenant rate limits and load shedding remove the regime rather than manage it. A rejected request with a retry-after hint is nearly always better than a request that waits four minutes: it returns control to the caller, it is measurable, and it prevents queue growth from converting a throughput problem into a latency outage. Pair this with **queue-time-based shedding** — drop work whose queue age already exceeds the client's timeout, since completing it produces no value while consuming capacity. This single rule prevents the classic collapse where a saturated pool spends all its capacity on requests nobody is waiting for anymore. ## Inside a tenant, be unfair Once cross-tenant isolation is in place, the intra-tenant policy should optimise throughput: unfair/barging locks, LIFO for cache warmth where latency permits, batching. All work in that bucket is equivalent to the customer, so ordering within it buys nothing and costs handoff latency. This layering — fair between classes, fast within a class — is the general shape of a good answer. ## What to measure Aggregate throughput and mean latency are the two metrics that will hide every fairness failure, so instrument: 1. **Per-tenant p99 and maximum wait time.** Maximum is the one that diverges under starvation. 2. **Age of the oldest queued item, per tenant.** The most diagnostic single number for a queue; it grows without bound under starvation while queue length may look stable. 3. **Achieved share versus configured weight.** If tenant A is configured for 20% and receives 3% under load, the policy is not doing what the config claims — usually a cost-accounting bug. 4. **Rejection/shed rate per tenant**, so you can see whether isolation is protecting or simply denying. 5. **Aggregate throughput and utilisation**, as the cost side: every point of fairness you buy shows up here, and you must be able to state the exchange rate. 6. **Behaviour under a deliberate noisy-neighbour test.** Run a load test where one tenant misbehaves — a hot loop of expensive queries — and verify the others' p99 stays within contract. A fairness policy that has never been tested under an adversarial tenant is a hypothesis, not a control. ## Reviewing the decision Re-evaluate when the tenant mix changes (a new tenant an order of magnitude larger), when request cost distributions shift, or when a fairness incident occurs. Record the contract and the exchange rate in writing: "tenants are guaranteed 10% of the pool each; enforcing this costs about 8% aggregate throughput". That sentence is what makes the tradeoff reviewable later instead of folkloric.