Connectors & Thread Pools

## Three limits, three different queues A request reaching Tomcat crosses three gates, and each has its own attribute on the `<Connector>` element in `server.xml`. Candidates who know only `maxThreads` misdiagnose load problems, because the symptom differs sharply depending on which gate is the one saturated. ```xml <Connector port="8080" protocol="HTTP/1.1" maxThreads="200" minSpareThreads="10" maxConnections="8192" acceptCount="100" connectionTimeout="20000" /> ``` **maxThreads** is the maximum number of request-processing threads this connector will create — therefore the maximum number of requests being *executed* at one instant. The default is 200. Each thread is named `http-nio-8080-exec-N`, which is how you recognise them in a thread dump. Note that if the connector references a shared `<Executor>`, the connector's own `maxThreads` is ignored and the executor's value governs. **maxConnections** is the maximum number of connections the connector will accept and keep open at one time. On the NIO connector the default is 8192. This number is much larger than `maxThreads` on purpose (see below). **acceptCount** is the length of the operating system's accept queue — the backlog Tomcat asks for when it binds the socket — and it only matters once `maxConnections` has been reached and Tomcat has stopped accepting. The default is 100. The OS can impose its own ceiling on the requested backlog. **minSpareThreads** (default 10) is unrelated to capacity: it is the number of threads kept alive even when idle, so a burst does not pay thread-creation cost on every request. ## Why connections and threads are decoupled Older Tomcat used one thread per connection, which meant an idle keep-alive connection wasted a thread. The NIO connector instead has a small number of poller threads watching sockets for readable data; a request thread is taken from the pool only when a complete request is ready to run, and handed back when the response is written. That is why `maxConnections` can sensibly be forty times `maxThreads`: ten thousand mostly-idle browser connections cost sockets and buffers, not threads. The practical consequence is that **thread starvation is invisible from the outside as an error**. If all 200 exec threads are blocked on a slow database call, connection 201 is still accepted and read — it simply waits. Clients experience growing latency; a proxy in front eventually gives up on its own timeout and reports a gateway error, while Tomcat's access log shows nothing at all until the request finally completes, because Tomcat writes the log entry after the response. ## The journey of one connection as load climbs 1. Under normal load: connection accepted, request read, an exec thread runs the servlet, response written, connection kept alive for the next request. 2. All exec threads busy: the connection is still accepted, but the parsed request waits internally for a free thread. Symptom: latency rises, error rate stays zero. 3. `maxConnections` reached: Tomcat stops calling accept. New connections pile up in the kernel accept queue. Symptom: connect succeeds (the OS completes the handshake) but nothing is read for a long time. 4. Accept queue full (`acceptCount` exceeded): the OS refuses or silently drops the connection. Symptom: connection refused, or connect timeouts — this is the first point at which the failure looks like a *network* problem, which is exactly why the whole chain gets misdiagnosed. ## Sizing, and the shared Executor `maxThreads` should follow the request's blocking profile, not a folklore number. If a request spends 90% of its time waiting on a downstream call, threads are cheap concurrency and a larger pool helps up to the point the downstream falls over; if requests are CPU-bound, a pool far above the core count only adds context switching and queueing. Raising `maxThreads` never fixes a saturated dependency — it just points more concurrency at it. Several connectors can share one pool with `<Executor>`: ```xml <Executor name="tomcatThreadPool" namePrefix="catalina-exec-" maxThreads="200" minSpareThreads="25"/> <Connector executor="tomcatThreadPool" port="8080" protocol="HTTP/1.1"/> ``` Sharing saves threads but removes isolation: a slow endpoint on one connector can consume the pool that another connector depends on. Tomcat's executor also differs from a stock JDK thread pool in that it grows the pool up to `maxThreads` *before* queueing tasks, which is the behaviour you want for a request server. Finally, `acceptCount` is a shock absorber, not capacity. A large backlog converts refusals into long waits, which for a client that has already given up is worse than a fast failure — the request is executed by Tomcat after nobody is listening for the answer.