Thread Pools and Executors

Orderly shutdown stops accepting new submissions but drains the queue and lets running tasks finish. Immediate shutdown also refuses new work, discards the queue - returning the undone tasks - and signals running tasks to stop. Both requests return at once; termination happens later.

A delay is a lower bound, not an appointment: the task will not start before 100 ms, but it may start much later. The scheduler keeps pending tasks ordered by due time and can only run one when a worker is free and the clock has passed it.

Only 8 tasks can actually compute at once, so throughput is already capped by the cores. Extra workers add context switches, cache pollution, memory for stacks and more lock contention. Throughput flattens then dips, and per-task latency rises because more work is in flight.

The waiting parent still occupies a worker. If every worker is a blocked parent, no worker is left to run the queued children, so parents wait on children that can never start. It is a resource deadlock where the scarce resource is the worker thread.

Most runtimes exit only when every ordinary thread has ended. Idle pool workers park waiting for tasks forever, so a pool that is never shut down keeps the process alive. Marking workers as background threads lets the process exit, but then they are killed abruptly mid-work.

Refuse = fastest, most honest signal; needs a caller that can handle it. Drop-newest sheds load and keeps old work; drop-oldest keeps the freshest data. Block propagates backpressure but can stall the producer. Run-inline throttles the producer by making it do the work.

An unbounded queue never rejects, so overload becomes unbounded memory growth and unbounded latency instead of a visible failure. Bounding it turns overload into an immediate, observable signal you can shed, retry, or push back on.

Fixed-rate anchors each run to a fixed timeline from the first start, so run count over time is preserved and execution time eats into the gap. Fixed-delay measures the gap from the end of the previous run, so the actual period is interval plus execution time and drift accumulates.

Most schedulers cancel a repeating task permanently if one run throws an exception that escapes it, and the failure is only recorded in a result handle nobody inspects. Defend by catching everything inside the task body and monitoring last-success time, not just process health.

A worker is runnable only the fraction S/(S+W) of its life, where S is compute time and W is wait time. To keep C cores busy at target utilization U you need N = C x U x (1 + W/S) workers. CPU-bound means W is zero, so N is about the core count.

Check whether anything is completing. Saturation still finishes tasks - slowly - and drains when load falls. Deadlock finishes nothing: sample worker stacks twice; identical stacks on the same tasks, with a flat completed-task counter, means stuck. Saturation needs load control; deadlock needs a structural fix plus restart.

Cancellation is cooperative: a request sets a flag or interrupt on the task's thread, and the task must observe it - checking between units of work and using blocking operations that wake on the signal. A task that never checks, or blocks in an operation that ignores it, cannot be stopped.

While the submitter runs the task itself it cannot submit anything else, so the intake rate is throttled to the pool's drain rate automatically. The risks: whatever else that thread was responsible for stalls — accepting connections, an event loop, ordering guarantees — and if the task needs the same pool, it can deadlock.

Direct handoff has zero buffer: a submission either starts now or fails now, so pressure is felt instantly and latency is not hidden. Buffering absorbs bursts but delays that signal — and in pools that only add workers when intake fails, a deep buffer means the pool never grows.

Schedulers serialize a periodic task rather than overlapping it, so missed occurrences pile up and each run starts immediately after the last — effectively continuous execution with no idle gap. Fix by measuring duration, skipping missed occurrences, bounding each run, and switching to a gap-based schedule.

Little's law: in a stable system the average number of items present equals arrival rate times average time in the system, L = lambda x W. Busy workers needed is arrival rate times service time. Response time grows like 1/(1 - utilization), so the last few percent cost enormous latency.

Partition so that one dependency's slowness cannot consume threads other work needs - a bulkhead per dependency or per criticality tier. The cost is lost sharing: each pool must be provisioned for its own peak, so total threads and idle capacity rise and utilization falls.

Stop intake first, then shut pools down in reverse dependency order - producers before the consumers they feed - each with an explicit slice of the grace period, escalating from drain to abort when a slice expires. Reserve time at the end for flushing and reporting anything still running.

Decide the overload contract explicitly: bound the queue from a latency budget, separate work by criticality into its own pools, shed the lowest-value class first, attach deadlines so stale work is skipped, and make the refusal a signal producers act on with backoff and capacity planning.

One pool cannot have one correct size: the three classes have wildly different wait/service ratios, and slow tasks occupy workers so fast ones queue behind them. Split into pools sized per class, budget CPU across them, cap each class's concurrency, and measure utilization and queue wait per pool.

Decide per job whether a missed occurrence must be replayed, coalesced into one run, or skipped; make runs idempotent and keyed by the occurrence they represent. Use a monotonic clock for relative delays and an explicit time zone for calendar times, and treat a doubled or missing local hour as a fire-once decision.