In a single FIFO admission queue on an analytical warehouse, why does one long query delay hundreds of short ones?

## The shape of the problem Admission control limits how many queries run at once; the queue holds the rest. If that queue is a single FIFO line, the wait a query experiences depends entirely on **who arrived before it**. That is fine when all work is similar. Analytical workloads are the opposite: runtime distributions are heavy-tailed, spanning milliseconds (a pruned point lookup on a clustered table) to hours (a full rebuild of a fact table). Mixing those in one ordered line means the cheap majority routinely inherits the latency of the expensive minority. This is **head-of-line blocking**: the item at the head of a queue holds up everything behind it regardless of how quickly the followers could be served. It is the same phenomenon as one slow request stalling a pipelined connection, or one slow consumer stalling a shared partition. The user-visible symptom is distinctive and worth recognising: **dashboard latency becomes bimodal**. Most of the day the dashboard answers in two seconds; several times a day it answers in twenty minutes. Nothing about the dashboard query changed — only what was ahead of it. ## Why "just raise the limit" is the wrong fix The instinct is to admit more queries so the short ones slip past. That trades one pathology for another: raising global concurrency shrinks per-query memory grants, which pushes the large queries into spilling, which makes them hold their slots even longer. You have made the head of the line slower while adding contention. Concurrency limits and queue structure are separate knobs and must be tuned separately. ## Lanes, not a line The real fix is to stop having one queue. Split admission into **classes** (queues, resource groups, priority pools — the vocabulary differs by engine) where each class has: - its own concurrency limit and memory grant sized for the class's queries; - a **reserved** share of capacity, so a busy class cannot consume another's; - optionally a maximum runtime, after which the query is aborted or moved. A short-query class with a small grant and high concurrency, reserved capacity, and a hard runtime cap gives interactive users a lane no batch job can ever occupy. A batch class with a large grant and low concurrency gives the transforms the memory they need without spilling. Neither starves the other, because the reservation is enforced at admission. ## Getting queries into the right lane Classification can be done on: - **Identity** — the submitting user, role, service account or application name. Crude but robust, and usually the first thing implemented: BI service account goes to the interactive lane, the orchestrator's service account goes to batch. - **Predicted cost** — the optimizer's estimated bytes to scan or estimated rows. Cheap to obtain (it comes out of planning) but only as good as the statistics; a stale estimate misroutes. - **History** — a normalised query fingerprint plus the runtime of previous executions of that fingerprint. Very accurate for the repeating dashboard and report traffic that dominates warehouses, useless for genuinely novel ad-hoc SQL. Most production setups combine them: route by identity first, override by predicted cost, and correct with history. ## Handling misclassification Every classifier is wrong sometimes, and the dangerous error is a large query admitted into the short lane, where it blocks exactly the traffic the lane exists to protect. Two mechanisms handle it: - **Runtime caps with demotion**: when a query in the interactive class exceeds its allowed runtime, move it into the batch class and let it continue there. The interactive slot is freed; the query still completes, though slower. This is far kinder than killing it. - **Abort rules**: for classes where nothing should ever run long — a public-facing dashboard lane, for instance — cancel outright and surface a clear error, so the author fixes the query rather than retrying it. ## The costs of lanes Reservations mean a lane's capacity can sit idle while another lane queues, so total utilisation falls. Most engines soften this by letting an idle lane's share be borrowed and reclaimed when its own work arrives; where that is not available you are explicitly buying isolation with utilisation, which is usually the right purchase for interactive SLAs. The second cost is **starvation of the long tail**. If the interactive lane always has work and batch is served only from leftovers, the nightly transforms never finish. Guard against it with a floor for the batch class, or with aging — raising a query's effective priority the longer it has waited — so no query can be postponed indefinitely.