A service runs three kinds of work on one shared worker pool: fast in-memory lookups, slow calls to a third-party HTTP API, and periodic report generation. Make the case for splitting them into separate pools, and explain how you would allocate capacity across the pools.

## Why one pool cannot be sized Pool size follows from the ratio of waiting to computing. In-memory lookups are almost pure service time, so their pool wants roughly the core count. Third-party HTTP calls may wait a hundred times longer than they compute, so their pool wants many times the core count. Report generation is CPU-heavy, long-running, and bursty. A single pool must pick one number, and every choice is wrong for two of the three classes: small enough for the lookups starves the HTTP work; large enough for the HTTP work oversubscribes the CPU whenever reports and lookups are active. ## Head-of-line blocking and the tail With one queue and one set of workers, task ordering couples the classes. A burst of report tasks or slow API calls occupies the workers; the next lookup, which needs a millisecond, waits behind them. The lookup's latency distribution becomes a mixture dominated by the slowest class - a textbook tail-latency amplifier. Fairness within a single pool does not fix it, because the unit of occupancy is a whole task: once a long task holds a worker it keeps it until it finishes. ## Isolation and correlated failure The strongest argument is failure containment. When the third-party dependency degrades from 50 ms to 5 s, its tasks stop leaving the pool. The arrival rate is unchanged and residence time is up a hundredfold, so by Little's law the number resident rises a hundredfold and consumes every worker. The lookup path - which needs nothing from that dependency - now fails too. Separate pools are bulkheads: the blast radius is one class. This is also why a per-class concurrency limit belongs on the calling side even when the executor is shared. ## Allocating capacity CPU is the genuinely shared resource; workers are just permits. 1. **Budget the CPU.** Sum the CPU demand of the CPU-bound classes and keep the total near the core count. Two pools of eight on eight cores means sixteen runnable threads competing - the isolation is real but the CPU is still shared, so protect it with priority or with smaller pools. 2. **Size the blocking class from its ratio, then cap it by the dependency.** The wait-ratio rule may say 200; if the third party degrades past 40 concurrent calls, 40 is the answer and the rest becomes visible queueing or fast rejection at your edge. 3. **Give background work a floor, not a share.** Reports get a small pool, lower priority, and a bounded queue; they are graded on completion, not latency, so running them at high utilization is correct. 4. **Decide the overflow policy per class.** Interactive classes want a shallow queue and fast rejection so callers can retry or degrade; background classes can queue deeply. A single global queue cannot express both. 5. **Consider elasticity.** For blocking classes a pool that grows to a hard ceiling under queueing pressure and shrinks when idle often beats a fixed number, provided the ceiling is set by the dependency. ## The cost of splitting Partitioning loses statistical multiplexing: three pools each sized for their own peak need more total capacity than one pool sized for the aggregate peak, because idle workers in one pool cannot help another. Fragmentation also multiplies the tuning surface and the number of places a change must be made, and every extra pool adds threads competing for the same cores. That is the honest counterargument, and the answer is not 'never split' but 'split along failure and latency boundaries, not along every function' - typically one pool per dependency or per latency class, not one per endpoint. ## How you would know it is right Per-pool telemetry: utilization, queue depth, queue wait, task duration percentiles, rejections. Two signals justify a split after the fact - a latency class whose percentiles track another class's activity, and a dependency incident whose impact extended beyond the tasks that used it. Two signals argue against over-splitting: pools persistently idle while another rejects work, and a total thread count far above what the cores can use. ## The framing to leave with Pool boundaries are an isolation decision first and a sizing decision second. The sizing math tells you how big each pool should be; the partition itself is chosen so that one workload's bad day cannot become another's.