skip to content

What is a poison-pill record, and how does ErrorHandlingDeserializer solve it?

level: seniorimportance: must knowfreq 70%

answer

  1. Undecodable record stalls partition forever
  2. Fails inside poll(), offset never advances
  3. ErrorHandlingDeserializer wraps a delegate
  4. spring.deserializer.value.delegate.class
  5. DeadLetterPublishingRecoverer -> DLT
  6. Error stored in record headers, payload null

basics

~20 s

A poison pill is a record whose bytes can't be deserialized, so it throws on every poll and blocks the consumer forever. ErrorHandlingDeserializer wraps the real deserializer, catches the failure, and hands the error to a recoverer instead of crashing.

solid answer

~40 s

A poison-pill record is one whose payload cannot be deserialized — corrupt bytes, wrong schema, or a serializer/deserializer mismatch. Because deserialization happens inside poll() and the offset never advances, the same record fails on every retry, stalling the partition indefinitely. Spring Kafka's ErrorHandlingDeserializer (org.springframework.kafka.support.serializer.ErrorHandlingDeserializer) wraps a delegate deserializer named via spring.deserializer.key.delegate.class / spring.deserializer.value.delegate.class. When the delegate throws, it catches the exception and returns null, stashing a DeserializationException in the record headers. Downstream, a DefaultErrorHandler (often with a DeadLetterPublishingRecoverer) inspects the header and routes the bad record to a DLT instead of looping. This converts an unrecoverable poll() failure into a per-record, recoverable event so the consumer keeps progressing.

code

properties · 4 lines
properties
key.deserializer=org.springframework.kafka.support.serializer.ErrorHandlingDeserializer
value.deserializer=org.springframework.kafka.support.serializer.ErrorHandlingDeserializer
spring.deserializer.key.delegate.class=org.apache.kafka.common.serialization.StringDeserializer
spring.deserializer.value.delegate.class=io.confluent.kafka.serializers.KafkaAvroDeserializer

go deeper

for a junior

Know the term: a bad record that the consumer can't decode and keeps choking on.

for a middle

Know that deserialization happens in poll() so the offset stalls, and that ErrorHandlingDeserializer + a DLT is the standard remedy.

for a senior

Configure the delegate classes, explain the header + null-payload mechanism, and wire DefaultErrorHandler/DeadLetterPublishingRecoverer.

for a principal

Design the DLT topology, distinguish deserialization vs business errors, handle null-ambiguity, and compare Spring vs Kafka Streams exception handling.

**The failure mode.** A Kafka consumer's `poll()` fetches a batch of raw records and *deserializes each one* before returning them. If one record's bytes are undecodable — truncated, written by an incompatible producer, encrypted, or simply the wrong format — the configured `Deserializer.deserialize()` throws (typically `SerializationException`). That exception propagates out of `poll()`. Crucially, the consumer's committed offset still points at the bad record, so the next `poll()` re-fetches and re-fails the *same* record. The partition is stuck forever. This stuck, repeatedly-failing record is the **poison pill**. **Why you can't just catch it.** The throw happens deep inside the client's fetch path, before your application code sees a `ConsumerRecord`. With the vanilla `KafkaConsumer` you have no hook to skip just that record except manually advancing the offset (`seek`) after catching the exception — error-prone and easy to get wrong. **ErrorHandlingDeserializer (Spring Kafka).** The fix is a *decorator* deserializer: `org.springframework.kafka.support.serializer.ErrorHandlingDeserializer`. You configure it as the actual `value.deserializer` (and/or `key.deserializer`), and tell it the real one via: - `spring.deserializer.value.delegate.class` - `spring.deserializer.key.delegate.class` When the delegate throws, `ErrorHandlingDeserializer` **catches** it, returns `null` for the value/key, and stores the captured `DeserializationException` (with the original bytes) in the `ConsumerRecord` headers (`ErrorHandlingDeserializer.VALUE_DESERIALIZER_EXCEPTION_HEADER`). poll() now succeeds; the bad record flows through as a record with a null payload plus an error header. **Completing the loop.** A null payload alone isn't a resolution. Spring's `DefaultErrorHandler` (and listener container) detects the deserialization-exception header and invokes a recoverer — commonly `DeadLetterPublishingRecoverer`, which republishes the original bytes to a dead-letter topic (DLT, e.g. `<topic>.DLT`) and lets the consumer commit and move on. Without a recoverer, the container can still log/skip rather than loop. **Plain Apache Kafka equivalents.** Outside Spring, you handle this with a custom deserializer try/catch, or with Kafka Streams' `DeserializationExceptionHandler` (`default.deserialization.exception.handler`, e.g. `LogAndContinueExceptionHandler` vs `LogAndFailExceptionHandler`). Confluent's schema-registry deserializers also throw `SerializationException` on schema problems. **Edge cases.** (1) A `null` value is ambiguous — it can mean a legitimate tombstone or a swallowed deserialization error; consumers must check the error header, not just the null. (2) ErrorHandlingDeserializer only handles *deserialization* failures; business-logic exceptions in your listener are a separate concern (retry/backoff/DLT via the error handler). (3) For keys, a failed key deserialization still lets you DLT the record but you lose key-based partitioning semantics.

  • After ErrorHandlingDeserializer catches the error, how does the bad record actually leave the main flow?
    It returns null and adds a DeserializationException header. A DefaultErrorHandler detects that header and a recoverer such as DeadLetterPublishingRecoverer republishes the original bytes to a dead-letter topic, then the offset commits so the consumer advances.
  • Why can't a normal try/catch around your listener fix a poison pill?
    Deserialization happens inside poll(), before your listener runs. The exception never reaches your code as a ConsumerRecord; poll() itself throws, so a listener-level try/catch is too late.
  • How is this handled in Kafka Streams instead of Spring?
    Via default.deserialization.exception.handler — LogAndContinueExceptionHandler skips bad records, LogAndFailExceptionHandler stops the stream. You can also supply a custom DeserializationExceptionHandler.

saying these in an interview costs you the question

  • Suggesting a try/catch around the @KafkaListener method fixes it — too late; poll() already threw
  • Saying the consumer 'just skips' bad records by default (it does not; it loops forever)
  • Treating a null value as always a tombstone when ErrorHandlingDeserializer also yields null on failure
  • Believing ErrorHandlingDeserializer handles business-logic exceptions (it only handles deserialization failures)

context

resolving links…