On a Tomcat service, one endpoint spends nearly all of each request blocked on a slow third-party API and its exec threads are saturated. How do you decide between raising maxThreads, converting the endpoint to an async servlet, and giving it its own <Executor> or connector?

## Name the constraint before choosing the lever The three options answer three different questions, and picking one before you know which question you are in is the classic failure here. - **maxThreads** answers *how much concurrency may execute at once*. - **async** answers *must a container thread be held while we wait*. - **a separate Executor or connector** answers *who suffers when this endpoint misbehaves*. So the first job is to characterise the wait. A thread dump tells you whether the exec threads are blocked in a socket read (waiting on the third party), waiting for a connection from a client pool (the pool is the real ceiling), or running (a genuine CPU constraint, where none of these three options helps). The distinction between *slow but working* and *failing* matters as much: for a dependency that is timing out or erroring, the correct response is a bounded timeout and a circuit breaker, and every capacity lever below merely increases the volume of work you throw at something already broken. ## What async actually buys, and what it costs With `request.startAsync()`, the servlet registers a callback, returns, and the exec thread goes back to the pool while the response is still pending. The request is completed later from whatever thread the asynchronous client uses. ```java @WebServlet(urlPatterns = "/quote", asyncSupported = true) public class QuoteServlet extends HttpServlet { @Override protected void doGet(HttpServletRequest req, HttpServletResponse resp) { AsyncContext ctx = req.startAsync(); ctx.setTimeout(3000); upstreamClient.fetchAsync().whenComplete((body, err) -> { try { ctx.getResponse().getWriter().write(body); } catch (Exception e) { /* handle */ } finally { ctx.complete(); } }); } } ``` The gain is real and structural: 200 exec threads no longer cap you at 200 concurrent in-flight requests. But it comes with conditions people routinely miss. - The whole path must be non-blocking. If the async callback calls a *blocking* HTTP or JDBC client on some other pool, you have moved the queue, not removed it. - Every filter in the chain must support async, and the servlet must declare `asyncSupported`. - Debugging and tracing get harder: the stack that fails is no longer the stack that received the request. - Removing the thread ceiling removes a **natural backpressure mechanism**. A saturated pool used to reject new work implicitly by making it wait; without it, you can accept vastly more concurrent requests than the downstream can serve, and turn a slow endpoint into an outage for the third party too. Async needs an explicit concurrency bound to replace the implicit one, plus a timeout — the connector's `asyncTimeout` defaults to 30000 ms, and `AsyncContext.setTimeout` overrides it per request. ## What isolation buys If the honest problem is that a marginal endpoint is starving the ones that pay the bills, isolation is the highest-value change and often the easiest to justify. A dedicated `<Executor>` used by a second connector — or simply a bounded worker pool inside the application for that call — means the failure consumes a defined slice. ```xml <Executor name="partnerPool" namePrefix="partner-exec-" maxThreads="30" minSpareThreads="5"/> <Connector executor="partnerPool" port="8081" protocol="HTTP/1.1"/> ``` This is the bulkhead idea applied at the container: you accept that the flaky endpoint will fail under stress, and you buy the guarantee that it fails alone. The trade is utilisation — reserved capacity sits idle when that endpoint is quiet — and operational complexity, because traffic now has to be routed to the right port by whatever sits in front. ## When raising maxThreads is legitimate It is not always wrong. If each request blocks briefly, the dependency has plenty of headroom, and per-thread cost is comfortable at the new number, more threads convert directly into more throughput and cost you one line of configuration instead of a rewrite. Be explicit about the two ceilings you are moving toward: the memory and scheduling cost of the threads themselves, and the concurrency the downstream will tolerate. Doubling `maxThreads` against a dependency that is already at its limit does not add throughput; it adds queueing and pushes the failure onto someone else's service. ## How to decide, and how to know you were right A defensible order of reasoning: *bound the call* (timeout, circuit breaker) → *contain the damage* (isolate the pool) → *remove the artificial ceiling* (async, with an explicit concurrency limit) → *buy more capacity* (resize, having verified the downstream can take it). The first two are cheap and almost always correct; the third is a real engineering investment that should be justified by a measured ceiling; the fourth is a dial, not a design. Validate with numbers you agreed on beforehand: saturation as the ratio of busy exec threads to `maxThreads`, tail latency of the healthy endpoints during a slow-dependency episode, and the error rate of the third-party call. If tail latency on the unrelated endpoints stops moving when the third party degrades, the isolation worked — regardless of whether the slow endpoint got faster, which was never the goal.