Virtual threads changed the calculus for blocking I/O and not for CPU work

A service handles 200 concurrent requests on a 200 thread pool. Each request makes three sequential HTTP calls to internal services, totalling around 400 milliseconds of waiting and maybe 5 milliseconds of actual work.

At 200 concurrent requests, every thread is occupied and the queue starts growing. Raising the pool to 800 threads means 800 megabytes of stack reservations and enough scheduler pressure that context switching becomes visible.

The threads are not busy. They are parked in a socket read, holding a megabyte of stack each so they can do nothing efficiently.

Virtual threads exist for exactly this, and the reason they help is narrow enough to be worth stating precisely.

The cost being removed is waiting, not working

A platform thread maps to an OS thread. It reserves around a megabyte of stack, the kernel schedules it, and switching between them means a trip through the kernel.

A virtual thread is a Java object on the heap. It runs on a carrier thread drawn from a small ForkJoinPool sized to the core count. When it blocks on something the JVM understands, its stack is copied to the heap and the carrier is released to run something else.

flowchart TB
    subgraph platform["Platform threads"]
        P1["Thread 1: parked in socket read<br/>1MB stack, OS scheduled"]
        P2["Thread 2: parked in socket read<br/>1MB stack, OS scheduled"]
        P3["800 threads = 800MB of waiting"]
    end

    subgraph virtual["Virtual threads"]
        C["Carrier pool: 8 threads"]
        V1["10,000 virtual threads<br/>blocked ones are heap objects"]
        V1 -.->|"mount only when runnable"| C
    end

    style P3 stroke:#ef4444,stroke-width:3px,color:#fff
    style V1 stroke:#4ade80,stroke-width:2px,color:#fff

The important consequence is that thread-per-request becomes viable again. Code written in plain blocking style, which is easy to read, easy to debug and produces useful stack traces, scales to concurrency levels that previously forced a reactive rewrite.

// One virtual thread per task. Blocking calls are fine here.
try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (Order order : orders) {
        executor.submit(() -> {
            var customer = customerClient.fetch(order.customerId());  // blocks
            var shipping = shippingClient.quote(order);               // blocks
            return enrich(order, customer, shipping);
        });
    }
}

Ten thousand of those is unremarkable. The same code on platform threads would need a pool, and the pool would be the bottleneck.

It does nothing for computation

This is the part that gets lost, and it follows directly from the mechanism.

Unmounting only helps if the thread is waiting. A thread that is computing is not waiting, so it holds its carrier for the full duration. A thousand virtual threads doing pure computation get the throughput of your core count, exactly as a thousand platform threads would, plus the overhead of mounting and unmounting.

Worse, a CPU bound virtual thread occupies a carrier that every other virtual thread needs. The carrier pool defaults to the number of available processors, so on an 8 core container, 8 virtual threads doing sustained computation starve every other virtual thread in the process. That is the same event loop failure that a CPU bound handler causes in an async runtime, arriving in a different language with different vocabulary.

CPU bound work belongs in a bounded pool of platform threads sized to the cores, deliberately separate from where the I/O work runs.

Pinning: where the abstraction leaks

A virtual thread cannot always unmount. When it cannot, it pins its carrier, and the carrier is unavailable to anyone else for the duration.

The two causes worth knowing are blocking inside a synchronized block, and blocking inside a native frame.

// The virtual thread cannot unmount while holding this monitor.
// The carrier is pinned for the entire network call.
public synchronized Response call() {
    return httpClient.send(request);      // pins for 200ms
}

// ReentrantLock is virtual thread aware. The carrier is released.
private final ReentrantLock lock = new ReentrantLock();

public Response call() {
    lock.lock();
    try {
        return httpClient.send(request);
    } finally {
        lock.unlock();
    }
}

With 8 carriers, 8 virtual threads pinned inside synchronized blocks doing network calls stops the entire process. This is a genuine deadlock risk that platform threads did not have, because a platform thread blocking in a synchronized block only blocks itself.

JDK 24 improved this substantially, allowing virtual threads to unmount while blocked inside synchronized, which removes the most common pinning source. On earlier versions it is a real constraint, and the practical advice is to replace synchronized with ReentrantLock on any path that performs I/O.

Finding it is straightforward:

-Djdk.tracePinnedThreads=full

That prints a stack trace whenever a virtual thread pins its carrier. Worth running in staging on any service adopting virtual threads, because the offenders are usually inside libraries rather than your own code.

Do not pool them

Pooling exists to amortise expensive creation. Virtual threads are cheap to create, so the pool provides nothing and takes something away.

// Wrong. Reintroduces the exact limit virtual threads remove.
ExecutorService bad = Executors.newFixedThreadPool(200, Thread.ofVirtual().factory());

// Right. One thread per task, no ceiling on concurrency.
ExecutorService good = Executors.newVirtualThreadPerTaskExecutor();

The fixed pool caps concurrency at 200 and queues everything else, which is what you were trying to escape.

That said, removing the ceiling means removing a limit that was doing real work. A pool of 200 was also limiting how many concurrent calls hit your database and your downstream services. Virtual threads let ten thousand requests proceed simultaneously, and ten thousand concurrent database queries is not a good outcome.

The limit still needs to exist, it just moves to where the constrained resource actually is: a semaphore around the downstream call, or the connection pool itself. That is the same reasoning that says pool size should be derived from what the dependency can absorb, and virtual threads make it more important rather than less, because the accidental limit is gone.

private final Semaphore downstream = new Semaphore(50);

public Response call() throws InterruptedException {
    downstream.acquire();          // explicit limit where it belongs
    try {
        return client.send(request);
    } finally {
        downstream.release();
    }
}

What this does and does not replace

It makes blocking code scale, which removes the main reason most teams adopted reactive frameworks. If the motivation for Reactor or RxJava was concurrency rather than composition or streaming semantics, virtual threads deliver it with code that is far easier to read and produces stack traces that mean something.

It does not remove the need to think about concurrency. Shared mutable state is still shared mutable state, races are still races, and an object that cannot change is still the cheapest way to avoid all of it.

It does not remove the need for backpressure. It arguably increases it, since the implicit limit that a thread pool provided is gone.

And it does not help thread-local heavy code. Ten thousand virtual threads each holding a thread-local cache is ten thousand copies of that cache, which is a memory problem that did not exist when there were 200 threads.

The honest summary is that virtual threads make one specific thing cheap: a thread that is waiting. That thing happened to be the dominant cost in most request-per-thread services, which is why the change feels larger than it is.