skip to content

In the servlet stack, how does SSE/emitter streaming interact with the threading model, and when does it stop scaling? How does WebFlux differ?

level: principalimportance: should knowfreq 30%

answer

  1. async frees request thread, not the connection
  2. socket/FD + worker thread per write
  3. no backpressure — producer buffers/blocks
  4. SimpleAsyncTaskExecutor default = unbounded trap
  5. WebFlux Netty event-loop + backpressure

basics

~20 s

Spring MVC frees the original request thread via async processing, but each open SSE connection still ties up a socket and needs a container thread whenever it writes. On a thread-per-connection container, thousands of concurrent long-lived streams exhaust the thread pool. WebFlux uses non-blocking I/O with a small event-loop, so it holds many idle streams cheaply.

solid answer

~50 s

When a controller returns an SseEmitter/ResponseBodyEmitter/StreamingResponseBody, Spring MVC puts the request into async mode: it commits headers and releases the original servlet request thread back to the pool, so the thread isn't parked for the stream's lifetime. But the response stays open — the OS socket is held, buffers are consumed, and every write borrows a container worker thread (and your producer runs on a task executor). The servlet model is fundamentally blocking/thread-per-request, so at high fan-out (tens of thousands of concurrent SSE clients) you hit socket and thread-pool limits and memory for buffers/emitter registry. WebFlux (Reactor Netty) is non-blocking: returning Flux<ServerSentEvent> or Flux<T> with text/event-stream lets a small event-loop multiplex huge numbers of idle connections, and Reactor provides real backpressure. Choose MVC SSE for modest concurrency and a servlet codebase; choose WebFlux when you need massive concurrent streams or end-to-end reactive backpressure.

code

java · 21 lines
java
// Bound the async executor so streaming producers don't spawn unbounded threads.
@Configuration
public class AsyncStreamingConfig implements WebMvcConfigurer {
    @Override
    public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
        ThreadPoolTaskExecutor exec = new ThreadPoolTaskExecutor();
        exec.setCorePoolSize(8);
        exec.setMaxPoolSize(32);
        exec.setQueueCapacity(100);
        exec.setThreadNamePrefix("sse-");
        exec.initialize();
        configurer.setTaskExecutor(exec);              // used by StreamingResponseBody / Callable
        configurer.setDefaultTimeout(120_000);          // 2 min default async timeout
    }
}

// WebFlux equivalent endpoint (non-blocking, backpressured):
// @GetMapping(value="/stream", produces=MediaType.TEXT_EVENT_STREAM_VALUE)
// Flux<ServerSentEvent<String>> stream() {
//     return source.map(v -> ServerSentEvent.builder(v).event("tick").build());
// }

go deeper

for a junior

Know SSE keeps a connection open and that many connections cost resources.

for a middle

Explain that async frees the request thread but the socket stays open and writes use container threads.

for a senior

Configure a bounded async executor, reason about slow clients, FD/connection limits, and proxy buffering.

for a principal

Trade off servlet SSE vs WebFlux backpressure at scale, design multi-instance fan-out via shared pub/sub, and set capacity/timeout/heartbeat policy deliberately.

**Servlet async recap.** Traditional Spring MVC is **thread-per-request**: a container worker (Tomcat) handles a request start-to-finish. Streaming return types (`SseEmitter`, `ResponseBodyEmitter`, `StreamingResponseBody`) trigger **Servlet 3.0 async processing** (`AsyncContext`). Spring commits the response early and **returns the original request thread to the pool** — crucial, because otherwise a long-lived stream would pin a worker thread for minutes/hours and you'd exhaust the pool almost immediately. So async solves the *initial* thread-pinning problem. **But the connection still costs.** Async does **not** make the connection free: - The **OS socket / file descriptor** stays open for the stream's whole life. Descriptor limits (`ulimit -n`) and connector `maxConnections` cap total concurrent streams. - Each **write** (`emitter.send`, StreamingResponseBody writing) is a **blocking I/O** operation that borrows a container worker thread for its duration. If a slow client can't drain, that write blocks and holds a thread — a slow-consumer can tie up threads. - Your **producer** (scheduler, @Async, listener) runs on a task executor; misconfigured, it defaults to the unbounded `SimpleAsyncTaskExecutor`, which spawns a thread per task — a real production trap. Configure `WebMvcConfigurer.configureAsyncSupport(...)` with a bounded `ThreadPoolTaskExecutor`. - Memory: the emitter **registry** plus per-connection buffers grow linearly with clients. **Where it stops scaling.** For modest concurrency (hundreds, low thousands) servlet SSE is fine. As concurrent long-lived streams climb into the tens of thousands, you hit: worker-thread contention on writes (especially with slow clients), connector connection limits, FD exhaustion, and buffer memory. The blocking model means idle-but-open connections aren't truly free the way non-blocking ones are. **No native backpressure.** The servlet emitters have **no reactive backpressure**: if you produce faster than the client consumes, data buffers (memory grows) or writes block. You must throttle producers manually. **WebFlux contrast.** Spring WebFlux runs on a **non-blocking** stack (Reactor Netty by default) with a small **event-loop** thread pool. A streaming endpoint returns `Flux<ServerSentEvent<T>>` (or `Flux<T>` produced as `text/event-stream`), and the event loop multiplexes **thousands of idle connections without a thread each** — a connection only consumes CPU when there's actual I/O. Reactor provides **end-to-end backpressure**: the subscriber signals demand, so a slow client naturally slows the source instead of buffering unbounded. This is the right tool for massive fan-out realtime streams. WebFlux SSE example: ```java @GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE) Flux<ServerSentEvent<String>> stream() { return Flux.interval(Duration.ofSeconds(1)) .map(i -> ServerSentEvent.<String>builder().id(i.toString()).event("tick").data("n=" + i).build()); } ``` **Nuance — MVC can adapt reactive types.** Spring MVC (servlet stack) can *return* `Flux`/reactive types via the `ReactiveAdapterRegistry`, but it still runs on the blocking servlet container underneath — you don't get Netty's event-loop efficiency; it's a bridge, not true reactive I/O. To get non-blocking scaling you must run the WebFlux stack (Netty). **Decision guidance.** - Existing servlet app, modest concurrent streams, simple push (progress bars, admin dashboards, LLM token stream to a handful of users) → **MVC SseEmitter** — simpler, familiar, blocking DB access is fine. - Very high concurrent connection counts, need backpressure, already reactive data sources → **WebFlux `Flux<ServerSentEvent>`**. - Bidirectional/binary realtime → **WebSocket** (either stack), not SSE. **Other production concerns (both stacks).** Reverse proxies/load balancers may buffer or idle-timeout streams — disable response buffering for the SSE path (e.g. nginx `proxy_buffering off`), send heartbeats, and set sensible `retry`. HTTP/1.1 browsers cap ~6 connections per origin (SSE eats one); HTTP/2 multiplexing relaxes this. Behind multiple instances, an emitter registry is per-instance — you need a shared pub/sub (Redis, broker) to fan an event out to whichever instance holds a given client's connection, and sticky routing or a message bus to reach the right node.

  • If async already returns the request thread, why does SSE still limit concurrency in the servlet stack?
    Because the connection itself persists: an open socket/file descriptor per client, blocking writes that each borrow a worker thread (worse with slow clients), and per-connection buffer memory. Non-blocking WebFlux avoids the thread-per-active-write cost.
  • You run three backend instances behind a load balancer. A domain event must reach a user whose SSE connection is on instance B. How?
    The emitter registry is per-instance/in-memory, so you need a shared pub/sub (Redis/broker): every instance subscribes; the producing instance publishes the event, and whichever instance holds that user's emitter delivers it. Sticky routing alone isn't enough for server-originated events.

saying these in an interview costs you the question

  • Saying async processing makes SSE connections free / infinitely scalable on the servlet stack.
  • Claiming Spring MVC SSE provides reactive backpressure.
  • Believing returning Flux from a servlet-stack controller gives Netty-level non-blocking scaling (it's an adapter, still blocking underneath).
  • Assuming an in-memory emitter registry works unchanged across multiple instances.

context

resolving links…