Virtual Threads: Motivation & Model

## The starting point: what a thread is A **thread** is an independent path of execution within a program. Each thread has its own **stack** (the memory holding local variables and the chain of in-progress method calls) and a **program counter** (where it is in the code). Multiple threads let a program do several things at once. ## Platform threads = OS threads (the old model) Before Java 21, every Java thread was a **platform thread**: a thin wrapper over an **operating-system (OS) thread**, mapped **one-to-one**. The OS thread is the real unit the operating system's **scheduler** (the OS component that decides which thread runs on a CPU core, and when) manages. OS threads are **expensive**: - Each reserves a large fixed **stack** (commonly ~1 MB), so memory caps how many you can have. - Creating/destroying one is a **system call** (a request into the OS kernel) — relatively slow. - **Context switching** (the OS saving one thread's state and loading another's so they can share a core) costs CPU. In practice a JVM tops out around a **few thousand** platform threads. ## Thread-per-request and the scalability wall A classic server uses the **thread-per-request** model: each incoming request gets its own thread that runs the whole request top to bottom — query a database, call another service, format a response. This code is easy to write, read, and debug, because it's plain sequential code. Most server work is **I/O-bound**: the thread spends most of its life **blocked** — parked, waiting for the network or disk, using no CPU. With only a few thousand threads, you can only have a few thousand requests in flight, even though the CPU is mostly idle. The bottleneck is the **thread count**, not the hardware. That is the scalability wall. ## The async/reactive escape — and its tax To break the wall, developers moved to **asynchronous / reactive** programming (CompletableFuture, reactive streams, etc.). Here a small pool of threads is **never blocked**; instead of waiting, a task registers a **callback** to run when the I/O finishes, and the thread moves on to other work. This scales to many concurrent requests on few threads. The cost is **complexity**. Sequential logic shatters into chained callbacks/futures. You can't use ordinary loops, try/catch, or step-through debugging across an async boundary; **stack traces** become uninformative. This is the **'colored function' problem**: once a method is async, everything that calls it must also be async — the 'color' spreads through the codebase, and you can't freely mix blocking and non-blocking code. ## Loom's idea: make threads cheap instead **Project Loom** asked the inverse question: instead of avoiding threads, make them so cheap that thread-per-request scales. The result is the **virtual thread** (Java 21, JEP 444). A **virtual thread** is a thread scheduled by the **JVM**, not the OS. It is **not** permanently bound to an OS thread. To actually run, a virtual thread is **mounted** onto a **carrier thread** — a regular platform (OS) thread drawn from a pool. When the virtual thread hits a **blocking** operation (e.g. socket read), the JVM **unmounts** it: its stack is copied to the **heap** (the JVM's general-purpose memory region), and the carrier thread is freed to run other virtual threads. When the I/O completes, the virtual thread is **remounted** (possibly onto a different carrier) and resumes. Because an idle virtual thread costs only a little heap, you can have **millions**. ## The payoff You go back to writing **simple, blocking, sequential code** — readable, debuggable, with real stack traces — and still get the scalability that previously required async. The JVM does the unmount/remount dance for you, so a blocking call no longer wastes a precious OS thread.