What problem does ExecutorCompletionService solve, and how does it differ from collecting a List<Future> and calling get() on each?
answer
- Decouples completion order from submission order
- Wraps an Executor + internal BlockingQueue of completed Futures
- take() blocks, poll()/poll(timeout) don't
- Kills head-of-line blocking of the get() loop
- Loop a count of N, not over the futures; map Future->input if needed
basics
~20 sExecutorCompletionService hands you results in the order tasks finish, not the order you submitted them. You call take() to get the next completed Future, so a slow task can't block you from processing fast ones that already finished.
solid answer
~50 sIf you submit N tasks and store their Futures in a list, then loop calling get() on each in order, you block on the first Future until it completes — even if other tasks finished long ago — so your throughput is bounded by submission order. ExecutorCompletionService fixes this: it wraps an Executor and an internal completion queue. Each submitted task, as it finishes, places its Future on that queue. You then call take() (blocking) or poll() (non-blocking/timed) to retrieve the next *completed* Future, in completion order. This lets you process results as soon as any are ready — ideal when you want the first acceptable answer (and can cancel the rest), or want to pipeline N independent results through downstream work without head-of-line blocking. You still call get() on the returned Future to read the value or surface its exception; CompletionService just decouples 'which task' from 'in what order I consume it'.
code
java · 22 linesExecutorService pool = Executors.newFixedThreadPool(8);
CompletionService<Result> cs = new ExecutorCompletionService<>(pool);
// fan out
for (Source s : sources) {
cs.submit(() -> s.fetch()); // Callable<Result>
}
// consume in COMPLETION order, not submission order
try {
for (int i = 0; i < sources.size(); i++) {
Future<Result> f = cs.take(); // next finished task
Result r = f.get(); // ExecutionException unwrap if it failed
process(r);
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (ExecutionException e) {
handle(e.getCause());
} finally {
pool.shutdown(); // CompletionService does NOT own the pool
}go deeper
Can state that CompletionService returns results as tasks finish rather than in submission order.
Explains the head-of-line blocking of a get() loop and how take()/poll() consume completed Futures from an internal queue.
Describes the wrap-an-Executor design, completion-queue mechanics, the first-good-answer/invokeAny and pipelining use cases, and the Future->input mapping caveat.
Weighs CompletionService against CompletableFuture and structured concurrency (StructuredTaskScope), discussing composition, cancellation propagation, and when the lightweight option still wins.
## The head-of-line blocking problem Suppose you launch N independent tasks and want to handle each result as it becomes available. The naive approach: ```java List<Future<Result>> futures = new ArrayList<>(); for (Task t : tasks) futures.add(pool.submit(t::run)); for (Future<Result> f : futures) { process(f.get()); // blocks on THIS one } ``` The consumer loop visits Futures in **submission order**. If `futures.get(0)` corresponds to the *slowest* task, you sit blocked on it while tasks 1..N-1 may have finished seconds ago. Their results pile up unprocessed. This is **head-of-line blocking**: a slow item at the front stalls everything behind it. ## What CompletionService does `CompletionService<V>` is an interface; the standard implementation is **`ExecutorCompletionService<V>`**. You construct it over an existing `Executor`: ```java ExecutorService pool = Executors.newFixedThreadPool(8); CompletionService<Result> cs = new ExecutorCompletionService<>(pool); ``` Internally it holds a **completion queue** (a `BlockingQueue<Future<V>>`, by default a `LinkedBlockingQueue`). When you `cs.submit(task)`, it wraps your task so that **when the task finishes, its completed `Future` is automatically enqueued** onto that queue. You then consume: - **`Future<V> take()`** — blocks until *some* task has completed, then returns its Future (in completion order). - **`Future<V> poll()`** — returns a completed Future immediately, or `null` if none is ready (non-blocking). - **`Future<V> poll(timeout, unit)`** — waits up to the timeout. Now the consumer drains results **as they complete**: ```java for (int i = 0; i < tasks.size(); i++) { Future<Result> f = cs.take(); // next FINISHED task process(f.get()); // read its value (or surface its exception) } ``` Note you loop a fixed *count* (you submitted N, so you take N) rather than over the futures themselves, because you no longer know — or care — which one comes next. ## Why this matters: two classic uses 1. **First-good-answer (invoke-any style):** query several replicas/sources in parallel, take the first successful result, then `cancel(true)` the rest. CompletionService gives you the first completion immediately. (The JDK's `ExecutorService.invokeAny` is built on this idea.) 2. **Pipelined fan-out:** run N independent computations and feed each result into downstream processing the instant it's ready, maximizing overlap and overall throughput. ## Contrast with List<Future> + get() loop | Aspect | List<Future> + get() loop | CompletionService | |---|---|---| | Consumption order | Submission order | Completion order | | Head-of-line blocking | Yes (slow front task stalls others) | No | | Knows which task produced result | Yes (index) | No (just 'next done') — keep a map if you need it | | Non-blocking check | Manual isDone() polling | poll()/poll(timeout) built in | ## Caveats - **Exceptions still surface via `get()`** on the returned Future (ExecutionException wrapping the cause); CompletionService doesn't change error handling. - If you need to know *which* input produced a result, store a `Map<Future, Input>` at submit time, since `take()` only hands you the Future. - CompletionService **does not** manage thread-pool lifecycle — you still own and shut down the underlying Executor. - It does not deduplicate or limit concurrency beyond what the wrapped Executor allows. ## Modern note `CompletableFuture` (Java 8) and structured concurrency (`StructuredTaskScope`, Java 21+) offer richer composition. But `ExecutorCompletionService` remains the simplest, allocation-light way to consume a batch of `Callable`/`Future` results in completion order.
- With ExecutorCompletionService, how do you know which original input produced the Future returned by take()?take() only gives you the Future, not the input. If you need the mapping, record it at submit time, e.g. Map<Future<V>, Input>, and look it up when the Future comes back.
- Does ExecutorCompletionService create or own its thread pool?No. You pass it an existing Executor; it only adds the completion queue. You remain responsible for shutting that Executor down.
A List<Future> get() loop is a single checkout lane where you can only serve customers in arrival order — the person at the front fumbling for change holds up everyone. CompletionService is a 'next available' system: whoever finishes packing first is called next, regardless of who arrived first.
saying these in an interview costs you the question
- Claiming CompletionService returns results in submission order — it's completion order.
- Thinking it owns/creates its own thread pool.
- Forgetting you still call get() (and handle ExecutionException) on the returned Future.
- Iterating over the submitted futures instead of taking N times — you lose the completion-order benefit.
- Assuming take() tells you which input it corresponds to.