skip to content

How do you enable a batch @KafkaListener, and what changes about delivery and error handling compared to record mode?

level: seniorimportance: should knowfreq 38%

answer

  1. setBatchListener(true) / spring.kafka.listener.type=batch
  2. method takes List<...>, size <= max.poll.records
  3. one failure fails the whole batch
  4. BatchListenerFailedException(index) -> commit good prefix, DLT the bad one
  5. throughput vs coarse retry granularity

basics

~20 s

Set batchListener=true on the container factory; the listener method then takes a List of records (one poll's worth) instead of a single record. You process them together, and error handling/retry applies to the whole batch unless you use index-aware handlers.

solid answer

~40 s

A batch listener consumes all records returned by one poll() at once. Enable it with factory.setBatchListener(true) (or spring.kafka.listener.type=batch in Boot); the method signature becomes List<MyType>, List<ConsumerRecord<K,V>>, or List<Message<?>>, optionally with Acknowledgment and Consumer params. Batch size is bounded by max.poll.records. It boosts throughput for bulk operations (batch DB inserts) by amortizing per-record overhead. The tradeoff is error handling: a failure in one record fails the whole batch. The DefaultErrorHandler can be paired with a BatchListenerFailedException (carrying the failing index) so spring-kafka splits the batch, commits the good prefix, and retries/DLTs only the bad record — otherwise the entire batch is reprocessed. AckMode MANUAL acks the whole list. Choose batch mode when work is naturally bulk and ordering/per-record retry granularity is acceptable.

code

java · 13 lines
java
factory.setBatchListener(true);

@KafkaListener(topics = "events", groupId = "ingest")
public void handle(List<ConsumerRecord<String, Event>> records) {
    for (int i = 0; i < records.size(); i++) {
        try {
            sink.write(records.get(i).value());
        } catch (Exception ex) {
            // commit the good prefix, retry/DLT only this record
            throw new BatchListenerFailedException("failed", ex, i);
        }
    }
}

go deeper

for a junior

Know batch mode passes a List of records per call and is enabled via the factory.

for a middle

Explain max.poll.records bounding and the throughput motivation.

for a senior

Detail BatchListenerFailedException, prefix commit, and the whole-batch failure default.

for a principal

Weigh batch sizing vs max.poll.interval.ms, memory, DLT strategy, and idempotency across the pipeline.

**Record vs batch dispatch.** A Kafka consumer's `poll()` returns *many* records at once (up to `max.poll.records`, default 500). By default spring-kafka iterates that collection and calls your `@KafkaListener` method **once per record**. In *batch* mode, the container instead hands the **whole list** to your method in a single invocation. **Enabling it.** Set `containerFactory.setBatchListener(true)` on the `ConcurrentKafkaListenerContainerFactory`, or `spring.kafka.listener.type=batch` in Spring Boot. The method signature must accept a collection: `List<Order>` (payloads), `List<ConsumerRecord<K,V>>` (full records with headers/offsets), or `List<Message<?>>`. You may also inject `Acknowledgment` (for MANUAL ack of the batch) and the raw `Consumer` (for seeks). **Why use it.** Throughput. If your downstream is itself batch-friendly — a JDBC `INSERT ... VALUES (...),(...)`, a bulk index into a search engine, a single network round trip — handling 500 records per call amortizes fixed costs far better than 500 separate calls. It also reduces commit overhead (one commit per batch). **How error handling changes — the key difference.** In record mode, the `DefaultErrorHandler` retries or dead-letters the *single* failing record and moves on. In batch mode, an exception thrown from the listener fails the *entire batch*, so the naive behavior is to reprocess all of it — including the records that already succeeded, causing duplicates. Spring-kafka solves this with **`BatchListenerFailedException`**: you throw it with the index (or the failing `ConsumerRecord`) of the record that failed. The `DefaultErrorHandler` then commits offsets for the successful *prefix*, and retries/DLTs only the offending record ("falling back to record mode" for that batch). Without it, the handler treats the whole batch as failed and applies the back-off/recovery to the entire list (`getRecords()` available on the thrown exception). **Acknowledgment.** With `AckMode.MANUAL`/`MANUAL_IMMEDIATE`, calling `ack.acknowledge()` commits the whole batch. Newer APIs allow partial acknowledgment of a batch via index. **Conversion.** Batch JSON conversion is supported via `BatchMessagingMessageConverter`; for typed `List<Order>` you typically rely on the value deserializer producing each element. **Edge cases and tradeoffs.** - A larger `max.poll.records` increases batch size but also the time between polls; if processing exceeds `max.poll.interval.ms`, the consumer is evicted and a rebalance occurs. - Per-record retry granularity is coarser; if you need precise per-record DLT routing, you must use `BatchListenerFailedException` discipline. - Memory: holding 500+ deserialized objects per thread per poll multiplies with concurrency. - Ordering within a partition is still preserved across batches. **Summary.** Batch mode trades fine-grained, automatic per-record error handling for throughput; production batch listeners almost always pair with `BatchListenerFailedException` + a `DefaultErrorHandler`/DLT to avoid wholesale reprocessing.

  • In batch mode, how do you avoid reprocessing already-successful records when one record in the batch fails?
    Throw BatchListenerFailedException pointing at the failing index/record. The DefaultErrorHandler commits the successful prefix and retries/dead-letters only that record, instead of replaying the whole batch.
  • What property bounds the size of each batch a listener receives?
    max.poll.records (consumer config, default 500) caps how many records one poll() returns, which is the upper bound of the list passed to a batch listener.

saying these in an interview costs you the question

  • Saying batch mode automatically retries individual records like record mode (it fails the whole batch by default)
  • Forgetting BatchListenerFailedException and thus replaying good records
  • Claiming batch size is unbounded (it's capped by max.poll.records)
  • Thinking batch mode changes per-partition ordering

context

resolving links…