skip to content

What is the difference between the acceptor thread and the network (processor) threads in a Kafka broker, and how many of each exist?

level: juniorimportance: should knowfreq 45%

answer

  1. 1 acceptor per listener, accept() only
  2. round-robin to processors
  3. num.network.threads default 3
  4. processors = NIO selectors, non-blocking I/O only
  5. no app logic in processors

basics

~20 s

The acceptor thread only accepts new client connections (one per listener). Network/processor threads handle reading and writing data on existing connections; there are num.network.threads of them (default 3), and the acceptor spreads new connections across them round-robin.

solid answer

~40 s

Per configured listener, the broker's SocketServer starts exactly one Acceptor thread whose sole responsibility is calling accept() on the server socket to accept new TCP connections. It then assigns each new connection round-robin to one of the Processor (network) threads. The number of processors is controlled by num.network.threads (default 3). Processors own NIO selectors and do all subsequent non-blocking I/O on their assigned connections: reading request bytes, parsing them and enqueuing them, and later writing responses back. So the acceptor is about connection setup (low volume), while processors are about per-request byte I/O (high volume) — which is why you typically have many more processors than acceptors.

go deeper

for a junior

Acceptor = accept new connections (one per listener); network threads = read/write data on connections (num.network.threads).

for a middle

Add round-robin assignment, NIO selector model, and connection affinity to a processor.

for a senior

Explain why processors must avoid blocking, and use NetworkProcessorAvgIdlePercent to decide tuning.

for a principal

Relate the reactor pattern and per-listener thread budgeting to capacity planning across many listeners/clusters.

Kafka brokers separate the act of *accepting* a connection from the act of *serving* it, mirroring a classic **reactor** networking pattern. ## Listener A listener is a host/port + security protocol the broker binds, e.g. `PLAINTEXT://broker1:9092`. A broker can have several (internal, external, controller). Each is configured via `listeners`. ## Acceptor thread For each listener, `SocketServer` creates **one** Acceptor. A TCP server socket can only be `accept()`ed by code somewhere; the acceptor is a tight loop doing exactly that and nothing else. Accepting is cheap and infrequent relative to request traffic, so one thread per listener suffices. After accepting a new `SocketChannel`, the acceptor hands it to a Processor chosen **round-robin**, balancing connections across processors. ## Processor (network) threads Controlled by **`num.network.threads` (default 3)**. Each Processor runs a Java NIO `Selector` event loop over the connections assigned to it. It performs: - **non-blocking** reads (accumulate bytes into a full request frame, parse into a `RequestChannel.Request`, enqueue on the shared request queue) - and **non-blocking** writes (drain its response queue back to sockets). Crucially, a Processor does **no application logic** — no disk, no waiting — so its event loop stays responsive and can multiplex thousands of connections. ## Why the asymmetry (1 acceptor vs many processors)? Connection establishment is a **one-time, lightweight event**; byte-level request/response I/O is **continuous and high-volume**. Scaling the part that does the heavy lifting (processors) while keeping the lightweight part (acceptor) minimal is efficient. ## Connection affinity Once the acceptor assigns a connection to a Processor, that connection stays on that Processor for its lifetime, including writing its responses. This avoids cross-thread coordination per connection. ## Tuning signal - If `NetworkProcessorAvgIdlePercent` is low (processors saturated), raise `num.network.threads`. - The acceptor almost never needs tuning. - Don't confuse processors with the request-handler/I/O threads (`num.io.threads`), which do the actual work after the request leaves the processor.

  • If you add a second listener, how many acceptor threads run?
    Two — one acceptor per listener. The processor pool size is still governed by num.network.threads (per listener configuration).
  • Which JMX metric tells you network/processor threads are saturated?
    NetworkProcessorAvgIdlePercent — values near 0 mean processors are busy and num.network.threads may need increasing.

saying these in an interview costs you the question

  • Saying processors do log reads/writes (that's the I/O / handler threads).
  • Thinking num.network.threads controls the acceptor count.
  • Claiming the acceptor reads request data — it only accept()s connections.

context

resolving links…