When advising a team migrating a thread-pool-based service to virtual threads, what design principles and pitfalls would you set as guidelines?
answer
- Mindset: limit the bottleneck (resource), not threads
- One VT per task; semaphore per downstream dependency
- I/O-bound only; CPU work stays on a core-sized pool
- Audit ThreadLocal -> ScopedValue for context
- Pinning: synchronized before 24, native frames always
basics
~20 sStop pooling virtual threads: one per task. Limit concurrency at the downstream resource with semaphores, not thread count. Use virtual threads only for I/O-bound work. Avoid heavy ThreadLocals; prefer ScopedValue. Watch pinning: synchronized on JDK 21-23, native calls always.
solid answer
~50 sThe core principle is that virtual threads remove the need to ration threads, so the design shifts from 'how many threads can I afford?' to 'where is the real bottleneck?'. Concretely: replace fixed thread pools with one-virtual-thread-per-task (newVirtualThreadPerTaskExecutor), and move concurrency limits from thread count to per-dependency semaphores sized to each downstream resource. Apply virtual threads to I/O-bound work only; keep CPU-bound work on a small pool sized to cores. Audit ThreadLocal usage — drop per-thread object caches (they assumed reuse) and migrate request context to ScopedValue. Beware pinning: on JDK 21-23, blocking inside synchronized tied a virtual thread to its carrier, so hot-path synchronized was swapped for ReentrantLock; JEP 491 removed that pinning in JDK 24, so 25 and 27 need no migration. Native and foreign frames can still pin. Keep blocking, imperative code rather than re-adding reactive layers, and watch carrier saturation and JFR pinning events.
code
java · 14 lines// Migration sketch: I/O on virtual threads, CPU work on a core-sized pool,
// and a semaphore per downstream dependency instead of a thread cap.
ExecutorService io = Executors.newVirtualThreadPerTaskExecutor(); // I/O work
ExecutorService cpu = Executors.newFixedThreadPool( // CPU work
Runtime.getRuntime().availableProcessors());
Semaphore apiLimit = new Semaphore(50); // bound the downstream API, not threads
Future<byte[]> result = io.submit(() -> { // returns a value, so a Callable
byte[] body;
apiLimit.acquire(); // may throw InterruptedException
try { body = callApi(); } // blocks; the virtual thread unmounts
finally { apiLimit.release(); }
return cpu.submit(() -> compress(body)).get(); // heavy compute on the CPU pool
});go deeper
Can list the headline rules: don't pool, one per task, I/O only, avoid heavy ThreadLocals.
Explains the rationale behind each rule (pooling re-caps concurrency, cores cap CPU work) and can apply them to a simple service.
Sequences a concrete migration — replace pools, add per-dependency semaphores, route CPU work separately, migrate context to ScopedValue — and handles pinning by JDK version: ReentrantLock on 21-23, nothing to migrate from 24.
Leads the whole effort: communicates the mindset shift, sets team conventions, designs backpressure and observability (carrier saturation, pinning events), weighs JDK-version behavior and library compatibility, and judges where reactive code should be retired versus retained.
## The mindset shift For two decades, JVM concurrency design has been dominated by one constraint: **OS threads are scarce and expensive**, so you ration them with **pools**. Virtual threads remove that constraint. The single most important thing to convey to a migrating team is the *mental model change*: stop asking *"how many threads can I afford?"* and start asking *"where is the actual bottleneck (a database, an API, the CPU)?"* and limit *there*. Most migration mistakes come from carrying pool-era reflexes into the virtual-thread world. ## Guideline 1 — One virtual thread per task; don't pool Pools amortize an expensive resource; virtual threads are cheap, so pooling them only **re-imposes the concurrency cap** you wanted gone and breaks `ThreadLocal` hygiene. Replace `newFixedThreadPool(n)` with `Executors.newVirtualThreadPerTaskExecutor()` (a *new* thread per submitted task, despite being an `ExecutorService`). Spawn freely. ## Guideline 2 — Move limits from threads to resources Unbounded concurrency is sometimes genuinely wrong — but the constraint is almost always a **downstream resource** (a DB connection pool, an API with a concurrent-call limit), not threads. Bound it with a **`Semaphore`** sized to that resource (acquire before the call, release in `finally`), and use a *different* semaphore per dependency so one slow service doesn't throttle everything. This decouples *how many tasks exist* from *how many may touch each resource*. ## Guideline 3 — Only I/O-bound work benefits Virtual threads scale by **unmounting when they block** on sockets, locks and queues, so a few carriers drive millions of waiting tasks. For **CPU-bound** work there is no blocking and parallelism is capped by **cores** — virtual threads add overhead, not speed. Keep CPU work on a small **fixed/fork-join pool** sized to `availableProcessors()`. Mixed services should route I/O work to virtual threads and CPU work to a bounded pool. ## Guideline 4 — Audit ThreadLocal; prefer ScopedValue With millions of one-shot threads, per-thread values **multiply memory** and the old trick of caching expensive objects in a `ThreadLocal` no longer pays off (it assumed thread *reuse*). Keep any remaining `ThreadLocal` small and immutable, and migrate **request-scoped context** to **`ScopedValue`** (JEP 506, final in Java 25): immutable, scope-bounded, auto-cleaned, and inherited without copying by **structured-concurrency** forks (`StructuredTaskScope`, a preview API in JDK 25 and 27). ## Guideline 5 — Watch for pinning, by JDK version A virtual thread normally unmounts when it blocks. **Pinning** is when it *can't* — it stays glued to its carrier, so a carrier is consumed while the thread waits, which can starve the (small) carrier pool. What pins depends on the release: - **JDK 21 to 23:** blocking inside **`synchronized`** pinned the thread. JEP 444's mitigation was to change `synchronized` code that runs often and guards long I/O to a **`ReentrantLock`**; code that runs rarely or guards in-memory work needed no change. - **JDK 24 and later (25 LTS, 27):** **JEP 491** lets virtual threads acquire, hold and release monitors independently of their carriers, so that migration is no longer necessary, and for new code JEP 491 recommends `synchronized` where practical. - **Still pinning after JEP 491:** its own list of remaining cases is a native method or foreign function (JNI/FFM) on the stack, for example native code that calls back into Java that blocks, blocking while loading a class, and blocking inside a class initializer. Keep long native calls off the hot path. - **Diagnostics:** the JFR event `jdk.VirtualThreadPinned` (enabled by default, 20 ms threshold) reports a thread that blocks while pinned. The `jdk.tracePinnedThreads` system property belongs to JDK 21 to 23 only: JEP 491 removed it in JDK 24, and setting it now has no effect. ## Guideline 6 — Don't over-engineer; keep it imperative A major *reason* to adopt virtual threads is to **delete** reactive/callback complexity. Resist re-introducing it. Straight-line **thread-per-request** blocking code is now both simple *and* scalable; readable stack traces and step-debugging return. For fan-out, **structured concurrency** (`StructuredTaskScope`) gives subtasks clear lifetimes and error propagation instead of ad-hoc futures, but it is still a preview API in JDK 21 to 27 and needs `--enable-preview`, so adopt it only where preview features are acceptable. ## Guideline 7 — Measure Validate the migration with observability: **carrier-thread saturation** (are carriers busy or starved by pinning?), **pinning events**, downstream **semaphore wait times**, latency/throughput. Right-size carriers (`jdk.virtualThreadScheduler.parallelism`) only if measurements justify it; defaults are usually fine. From JDK 24, `jdk.management.VirtualThreadSchedulerMXBean` reports the scheduler's parallelism, pool size and mounted and queued virtual threads over JMX. ## Summary Lead with the mindset shift (limit the bottleneck, not threads). Then: one VT per task, semaphores per dependency, VTs for I/O only, ScopedValue over heavy ThreadLocal, guard against pinning (synchronized before JDK 24, native frames on any release), keep imperative code (with structured concurrency where preview APIs are acceptable), and measure.
- What is pinning, why does it matter at scale, and how do you mitigate it?Pinning is when a virtual thread blocks but cannot unmount, so it holds its carrier. Carriers are few (by default one per available processor), so frequent, long pinning can starve them. On JDK 21-23, blocking inside synchronized pinned, and the fix was ReentrantLock on hot paths; JEP 491 removed that in JDK 24. Native or foreign frames and class loading can still pin; find them with the JFR event jdk.VirtualThreadPinned.
- How does structured concurrency complement these virtual-thread guidelines?StructuredTaskScope, still a preview API through JDK 27, ties forked subtasks to a parent scope with a clear lifetime, automatic cancellation and error propagation, replacing ad-hoc futures. Combined with one-VT-per-task, ScopedValue context inheritance and per-resource semaphores, it gives readable, leak-resistant fan-out concurrency, where preview features are acceptable.
- Migrating by 'swapping in a virtual-thread pool' of fixed size — keeps the old cap
- Putting CPU-bound work on unbounded virtual threads expecting a speedup
- Ignoring pinning (synchronized on JDK 21-23, native frames on any JDK) and starving carriers
- Rebuilding reactive/callback layers on top of virtual threads, losing the simplicity benefit
- Leaving expensive ThreadLocal caches in place, blowing up memory at scale
- Still setting -Djdk.tracePinnedThreads on JDK 24 or later, where it has no effect
- virtual threads
- Java 21pre-release 19
Assumes JDK 21 or later, where virtual threads are final after previews in JDK 19 and 20, and gives both pinning eras: synchronized pinned on JDK 21 to 23 and no longer does from JDK 24, so JDK 25 LTS and 27 behave the same; StructuredTaskScope is still a preview API at 27. Checked at JDK 27.
context
- Virtual Threadsdocs.oracle.com
- JEP 444: Virtual Threadsopenjdk.org
- JEP 491: Synchronize Virtual Threads without Pinningopenjdk.org