skip to content

Virtual Thread Usage Guidelines

The rules that change with virtual threads: do not pool them, limit concurrency with a Semaphore instead of a fixed pool, keep ThreadLocals light, and expect no benefit for CPU-bound work. Interviewers ask this to see whether you carried old thread-pool habits over unexamined.

part ofJavaread it →
on this pageshow

questions

5

Why should you NOT pool virtual threads, and what should you do instead?

level: middleimportance: must knowfreq 70%

answer

  1. Pools amortize expensive resources; VTs are cheap
  2. Fixed VT pool = re-imposes the old thread limit
  3. newVirtualThreadPerTaskExecutor = per task, not a pool
  4. Bound the resource with a Semaphore, not threads
  5. Pooling reuses threads -> ThreadLocal leaks across tasks

basics

~20 s

Virtual threads are cheap and disposable, so you create a fresh one per task instead of reusing them from a pool. Pooling exists to reuse expensive resources; virtual threads aren't expensive, so a pool only adds a bottleneck. Use Executors.newVirtualThreadPerTaskExecutor().

solid answer

~40 s

Thread pools exist to reuse a scarce, expensive resource: platform (OS) threads. A virtual thread is the opposite — it is cheap to create (a small heap object) and you can have millions of them, so reusing them buys nothing and reintroduces the very limit pools were meant to remove. Capping a virtual-thread pool to, say, 200 threads silently serializes your tasks back to 200-at-a-time, erasing the scalability win. The idiom is one virtual thread per task, via Executors.newVirtualThreadPerTaskExecutor() or Thread.ofVirtual().start(). If you must bound concurrency it should be against a downstream resource (a database, an API) using a Semaphore, not by limiting threads. Pooling also breaks ThreadLocal hygiene, since pooled threads carry state between unrelated tasks.

code

java · 21 lines
java
// WRONG: a fixed pool of virtual threads caps concurrency at 200
ExecutorService bad = Executors.newFixedThreadPool(200, Thread.ofVirtual().factory());

// RIGHT: one new virtual thread per task
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (var request : requests) {
        executor.submit(() -> handle(request)); // a new virtual thread each time
    }
} // close() waits for every submitted task

// RIGHT: still one per task, but bound the resource with a semaphore
Semaphore dbLimit = new Semaphore(50); // the DB pool has 50 connections
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (var request : requests) {
        executor.submit(() -> {         // returns a value, so it is a Callable,
            dbLimit.acquire();          // which may throw InterruptedException
            try { return queryDatabase(request); }
            finally { dbLimit.release(); }
        });
    }
}

go deeper

for a junior

Knows virtual threads are cheap and that you create one per task instead of reusing them, and can name newVirtualThreadPerTaskExecutor().

for a middle

Explains why pooling cheap threads is pointless and that a fixed pool silently caps concurrency back to the pool size, erasing the benefit.

for a senior

Articulates the resource-pooling rationale, distinguishes bounding threads from bounding a downstream resource with a Semaphore, and notes ThreadLocal leakage in pools.

for a principal

Frames it as a design principle (amortize scarce resources, not cheap ones), reasons about where backpressure truly belongs in the system, and can guide a team migrating an executor-pool codebase to per-task virtual threads safely.

## Background: what a thread is A **thread** is an independent path of execution. Traditionally, each Java thread is a **platform thread** — a thin wrapper over an **operating-system (OS) thread**. OS threads are *expensive*: each has its own stack and is created and scheduled by the OS, and JEP 444 puts it plainly: OS threads are costly, so we cannot have too many of them. In a thread-per-request server the thread count often becomes the limit long before the CPU or the network connections run out. ## Why thread pools exist Because platform threads are scarce and costly, the classic solution is a **thread pool** (e.g. `Executors.newFixedThreadPool(200)`): create a fixed set of threads *once*, then feed them a queue of tasks. The pool **reuses** each thread for many tasks so you pay the creation cost only once and never exceed a safe thread count. This is a form of **resource pooling** — the same reason we pool database connections. ## What a virtual thread is A **virtual thread** (final in Java 21 through JEP 444, after previews in 19 and 20) is a thread managed by the **JVM**, not the OS. Its stack lives in the Java heap and grows and shrinks as the code runs. Many virtual threads are multiplexed onto a small pool of **carrier** (platform) threads. When a virtual thread blocks in most JDK blocking operations (a socket read, `Thread.sleep`, `BlockingQueue.take()`), the JVM **unmounts** it from its carrier, freeing the carrier to run another virtual thread. Many file-system operations are the exception: an ordinary `FileInputStream` read, for example, holds the carrier for as long as it blocks. You can create **millions** of virtual threads. ## The key insight: don't pool what is already cheap Pooling is a technique to *amortize the cost of an expensive resource*. A virtual thread is **not** an expensive resource: JEP 444 calls virtual threads cheap to create and almost infinitely plentiful, and says there is never a need to pool them. Therefore: 1. **Pooling buys nothing.** There is no creation cost worth amortizing. 2. **Pooling reintroduces the bottleneck you were escaping.** If you put virtual threads behind a fixed pool of size N, only N tasks run concurrently — you have re-imposed the exact platform-thread limit virtual threads were designed to eliminate. With 10,000 concurrent I/O-bound requests and a pool of 200, 9,800 wait in a queue. 3. **Pooling corrupts thread-local state.** A pooled thread outlives one task and is reused, so any `ThreadLocal` value a task does not `remove()` is still there for the next, unrelated task. ## The correct idiom Create **one virtual thread per task** and let it die when the task finishes: - `Executors.newVirtualThreadPerTaskExecutor()` — an `ExecutorService` that spawns a brand-new virtual thread for every submitted task (it is *not* a pool — the name says "per task"). - `Thread.ofVirtual().start(runnable)` or `Thread.startVirtualThread(runnable)` for a one-off. ## But I still need to limit concurrency! Unbounded concurrency is sometimes wrong — not because of the threads, but because a **downstream resource** (a database with 50 connections, an API that accepts only so many concurrent calls) can't take 10,000 simultaneous calls. The right tool is a **`Semaphore`** sized to that resource, *not* a thread pool. Each virtual thread acquires a permit before touching the resource and releases it after; you still get one cheap virtual thread per task, but only N of them touch the constrained resource at once. This separates *how many threads exist* (as many as tasks) from *how many may use a resource* (bounded by the semaphore). ## Summary Pools amortize expensive resources; virtual threads are cheap, so pooling them only re-creates the limit you wanted gone and breaks `ThreadLocal` hygiene. One virtual thread per task; bound concurrency at the resource with a semaphore.

  • If unbounded virtual threads are fine, how do you protect a database that only has 50 connections?
    Keep one virtual thread per task, but guard the database call with a Semaphore(50). Each thread acquires a permit before the call and releases it after, so at most 50 hit the DB concurrently while the rest park cheaply — limiting the resource, not the threads.
  • Is Executors.newVirtualThreadPerTaskExecutor() a thread pool?
    No. Despite returning an ExecutorService, it creates a brand-new virtual thread for each submitted task and discards it on completion — there is no reuse. Its name literally says 'per task'.

  • Saying 'just make a pool of virtual threads sized large' — any fixed size still caps concurrency and defeats the purpose
  • Believing newVirtualThreadPerTaskExecutor is a reusing pool (it spawns a new thread per task)
  • Confusing limiting threads with limiting access to a downstream resource
  • Thinking pooling is needed to avoid 'too many' virtual threads when the real constraint is the downstream resource

virtual threads
Java 21pre-release 19
Executors.newVirtualThreadPerTaskExecutor
Java 21pre-release 19

Assumes JDK 21 or later, where virtual threads and Executors.newVirtualThreadPerTaskExecutor() are final after previews in 19 and 20; checked at JDK 27, and JDK 25 behaves the same.

context

open as a page

For which kinds of workloads do virtual threads help, and where do they give no benefit?

level: middleimportance: must knowfreq 65%

basics

~20 s

Virtual threads shine for I/O-bound work: tasks that mostly wait on the network, a database or other services. For CPU-bound work they give no benefit, because the limit is the number of CPU cores, not the number of threads.

open as a page

Your service spawns one virtual thread per request, but a downstream API that accepts only 50 concurrent calls starts failing under load. How do you bound concurrency correctly?

level: seniorimportance: should knowfreq 40%

basics

~20 s

Keep one virtual thread per request, but guard the downstream call with a Semaphore sized to what the API allows. Each thread acquires a permit before calling and releases it after, so only that many calls happen at once — without limiting how many threads exist.

open as a page

Why should you avoid heavy or expensive ThreadLocal use with virtual threads, and what's the alternative?

level: seniorimportance: should knowfreq 45%

basics

~20 s

Each virtual thread that uses a ThreadLocal gets its own value, and with millions of virtual threads, expensive per-thread values multiply into a lot of memory. So avoid large or mutable ThreadLocals; prefer ScopedValue for sharing read-only context.

open as a page

When advising a team migrating a thread-pool-based service to virtual threads, what design principles and pitfalls would you set as guidelines?

level: principalimportance: should knowfreq 35%

basics

~20 s

Stop pooling virtual threads: one per task. Limit concurrency at the downstream resource with semaphores, not thread count. Use virtual threads only for I/O-bound work. Avoid heavy ThreadLocals; prefer ScopedValue. Watch pinning: synchronized on JDK 21-23, native calls always.

open as a page