skip to content

On a database shared by many customer organizations, one large customer's reporting queries are slowing everyone else down. Explain the mechanisms by which one tenant degrades others in a shared relational database, and what you would do about it at the database and architecture level.

level: seniorimportance: should knowfreq 38%

answer

  1. shared buffer cache: one big scan evicts everyone
  2. connections and workers = fixed pool, starvation
  3. stats are per table, not per tenant -> plan skew
  4. timeouts + per-tenant quotas + replica for reports
  5. tag queries with tenant for attribution

basics

~20 s

Tenants share the buffer cache, CPU, I/O, connections and locks, so one heavy tenant evicts others' cached pages and starves workers. Fix with statement timeouts and per-tenant connection quotas, offload reporting to a replica, tune plans for skewed tenants, and move whales to their own database.

solid answer

~50 s

**Mechanisms:** a shared buffer cache (a big scan evicts everyone's hot pages), shared CPU and I/O bandwidth, a shared connection and worker pool (one tenant's burst starves others), temp space and memory for sorts and hashes, lock and latch contention on genuinely shared rows or sequences, and replication lag from one tenant's write burst, which degrades every reader on the replica. Table-level statistics describe an average tenant, so a whale can get a plan sized for a small tenant and scan far more than it should. **Fixes, cheapest first:** statement and idle-in-transaction timeouts; per-tenant connection and concurrency quotas enforced in the application or pooler; queueing expensive jobs with per-tenant fairness; moving reporting to a read replica or analytics copy; fixing plan skew for large tenants; and, where supported, resource groups or workload management to cap CPU and I/O per class. **Architecturally**, promote whales to their own database or pod - a shared instance has no strong resource boundary - and price that isolation tier.

go deeper

for a junior

Say that tenants share memory, CPU and I/O, so a big query from one customer slows the others, and that limits and timeouts help.

for a middle

Name the concrete shared resources - buffer cache, connections, temp space, locks - and propose timeouts, per-tenant quotas and moving reports to a replica.

for a senior

Add plan and statistics skew, replication lag, long-transaction bloat, per-tenant observability, and the case for promoting whales to their own database.

for a principal

Define isolation tiers with capacity and cost per tier, set the pooled-versus-siloed placement policy, and connect fairness guarantees to SLAs and pricing.

## Why one tenant can hurt all the others In a pooled or schema-per-tenant layout, tenants are logically separated but share every physical resource of the instance. Each of these is a channel through which one customer's workload becomes another customer's latency: **Buffer cache.** The engine keeps hot pages in memory. A single large sequential scan pulls gigabytes of one tenant's pages through the cache and evicts everyone else's working set; other tenants' formerly in-memory queries start hitting disk, and their latency rises by an order of magnitude long after the offending query ends. Some engines mitigate this with a small ring buffer for big sequential scans, but the effect is real. **CPU and I/O bandwidth.** Parallel workers and heavy scans consume cores and IOPS that everyone else queues behind. On cloud storage with burst credits, one tenant can exhaust the budget for the whole instance. **Connections and workers.** Connection slots and background or parallel workers are a fixed pool. A tenant that opens 200 connections - or whose retry storm does - starves others, and past a point more concurrency reduces total throughput because context switching and contention dominate. **Working memory and temp space.** Large sorts, hashes and materialized intermediates spill to temp files, competing for the same disk and sometimes filling it, which fails unrelated queries. **Locks and contention.** Genuinely shared objects - a counter table, a shared sequence, a global settings row, an index page every tenant appends to - serialize tenants against one another. Long-running reporting transactions also pin old row versions and block cleanup of dead rows, so bloat created by one tenant's open snapshot slows scans for everyone. **Replication lag.** A tenant's bulk import generates write volume the replica must replay; every tenant reading from that replica now sees stale data or waits. **Plan skew.** The optimizer holds statistics per table, not per tenant. With one tenant holding 60% of the rows, a plan chosen from average selectivity (say a nested loop expecting 100 rows) can be catastrophic for the whale at 10 million rows - and where plans are cached and shared, a plan compiled for a small tenant may be reused for the big one. ## What to do, in increasing order of cost 1. **Bound single queries.** Statement timeouts, idle-in-transaction timeouts and row limits stop one query monopolizing resources indefinitely. This alone converts most incidents from outages into failed reports. 2. **Quota concurrency per tenant.** Cap connections and in-flight expensive operations per tenant in the application or connection pooler, and rate-limit endpoints that trigger heavy queries. Fairness must be enforced above the database, because the database schedules per query, not per customer. 3. **Queue and schedule heavy work.** Route report generation and exports through a worker queue with per-tenant concurrency limits and off-peak scheduling, instead of running them synchronously on request. 4. **Separate the workload.** Send reporting to a read replica or an analytics copy shaped for it. This is the single most effective structural fix, because the transactional path stops competing with scans. The replica is itself shared, so a replica-level noisy neighbor is still possible - but customer-facing transactions are protected. 5. **Fix the skew.** Better indexes for the large tenant's access patterns, higher statistics targets where the engine supports per-column tuning, and avoiding one cached plan shared across wildly different tenants. 6. **Resource governance.** Where the engine offers it (resource groups, workload management, per-role I/O throttling), cap CPU, memory and I/O for a class of tenants. Coverage is uneven and rarely as strong as separation. 7. **Physical separation.** Move the whale to its own database or instance - a pod model where the biggest customers are siloed and the long tail stays pooled. This is the only mechanism that truly bounds resource impact, which is why isolation tiers usually appear in pricing. ## Detecting it You cannot manage this without **per-tenant observability**: tag queries with the tenant (a comment or the application name) so query statistics attribute time, I/O and rows to a customer. Track per-tenant query time, rows read and cache-hit ratio, and alert when one customer's share of instance resources crosses a threshold. Without attribution, incident response is guesswork; with it, the answer is usually one tenant and one query shape.

  • How would you identify which tenant is causing the degradation?
    Attribute work to tenants: tag every query with the tenant id in a comment or the session/application name, then aggregate the engine's query-statistics views by that tag to rank tenants by total time, rows read and I/O. Combine that with per-tenant application metrics and, during an incident, a snapshot of active sessions. Without attribution the instance only shows you slow queries, not the customer behind them.
  • Why does moving a large tenant to its own database help more than adding indexes?
    Indexes reduce that tenant's work but create no resource boundary: its remaining scans still share the buffer cache, CPU, I/O and connection pool. A separate database, and better a separate instance, gives an actual capacity ceiling, so bursts are contained and can be sized and billed independently. Indexing is the right first move; separation is the structural answer once one tenant's demand is a fixed fraction of the instance.

A shared instance is a shared kitchen: one household cooking a banquet uses every pot and the whole stove, so everyone else waits. Timeouts and quotas are house rules; giving the banquet cook their own kitchen is the only real fix.

saying these in an interview costs you the question

  • Claiming schema-per-tenant fixes noisy neighbors - the instance resources are still shared
  • Assuming the optimizer knows one tenant is a thousand times larger than another
  • Relying only on indexes while leaving heavy reporting on the primary
  • Solving it by raising the connection limit, which worsens contention
  • Having no per-tenant attribution and guessing at the culprit during an incident

context