What roles do ProducerFactory and ConsumerFactory play, and why configure serializers/deserializers there?
answer
- ProducerFactory -> KafkaTemplate; ConsumerFactory -> listener container
- producer cached/shared (thread-safe); consumer created per-thread (not thread-safe)
- serializers on producer factory, deserializers on consumer factory
- ErrorHandlingDeserializer for poison pills
- Boot auto-config from spring.kafka.*
basics
~10 sProducerFactory builds and caches KafkaProducer instances from a config map (bootstrap servers, serializers); KafkaTemplate uses it. ConsumerFactory builds KafkaConsumer instances (with deserializers) that listener containers use. Serializers/deserializers live there because they're producer/consumer-level settings.
solid answer
~40 sProducerFactory<K,V> and ConsumerFactory<K,V> are the spring-kafka beans that encapsulate the native client configuration and instantiate KafkaProducer/KafkaConsumer objects. DefaultKafkaProducerFactory holds the producer property map plus key/value Serializers and reuses a single thread-safe producer (or per-thread producers for transactions); KafkaTemplate is wired to it. DefaultKafkaConsumerFactory holds consumer properties and key/value Deserializers; the ConcurrentKafkaListenerContainerFactory uses it to create one consumer per concurrency thread. Serializers/deserializers belong here because (de)serialization is intrinsic to how the client reads/writes bytes. Common choices: StringSerializer, JsonSerializer, or Avro/Protobuf with Schema Registry. ErrorHandlingDeserializer wraps a delegate so a poison-pill payload becomes a handled error instead of killing the poll loop. Spring Boot auto-configures both factories from spring.kafka.* properties, but you define custom beans for per-binding serializers, trusted packages, or transactions.
go deeper
Know the factories create producers/consumers and hold serializer config.
Explain shared vs per-thread instances and where (de)serializers belong.
Discuss ErrorHandlingDeserializer, JSON trusted packages, transactions, multiple container factories.
Standardize serialization/schema strategy (Avro+registry), factory bean topology, and poison-pill resilience across services.
**The native clients.** Kafka's Java library produces with `KafkaProducer<K,V>` and consumes with `KafkaConsumer<K,V>`. Both are configured by a `Map<String,Object>` of properties (`bootstrap.servers`, `key.serializer`, `value.deserializer`, `group.id`, etc.) and convert between your domain objects and the bytes on the wire using *serializers* (object → bytes, producer side) and *deserializers* (bytes → object, consumer side). **ProducerFactory.** `DefaultKafkaProducerFactory<K,V>` stores the producer config map and the key/value `Serializer`s, and is responsible for **creating and managing** producer instances. Since `KafkaProducer` is thread-safe and expensive to create, the factory caches and shares a single producer by default (and creates per-`transactional.id` producers when transactions are enabled). `KafkaTemplate` delegates to this factory to obtain a producer for each send. You provide a custom `ProducerFactory` bean to set a non-default serializer, enable idempotence/transactions (`setTransactionIdPrefix`), or override `acks`/compression. **ConsumerFactory.** `DefaultKafkaConsumerFactory<K,V>` mirrors this for consumers: it holds consumer properties and key/value `Deserializer`s and **creates a fresh `KafkaConsumer` per request** — because `KafkaConsumer` is *not* thread-safe and each listener thread needs its own. A `ConcurrentMessageListenerContainer` with `concurrency=N` asks the factory for N consumers. **Why (de)serializers live in the factories.** Serialization is a per-client concern: the producer must know how to turn keys/values into bytes before they ever reach a topic, and the consumer must know how to reconstruct them. These are passed as constructor args or properties (`KEY_SERIALIZER_CLASS_CONFIG`, `VALUE_DESERIALIZER_CLASS_CONFIG`) on the factory, so every producer/consumer it builds is consistent. **Common serializer/deserializer choices.** - `StringSerializer`/`StringDeserializer` for text. - `JsonSerializer`/`JsonDeserializer` (spring-kafka) for JSON POJOs — the deserializer needs *trusted packages* (`spring.json.trusted.packages`) and/or default type mapping to avoid deserializing arbitrary classes. - Avro/Protobuf serializers backed by a Schema Registry for schema evolution. - `ErrorHandlingDeserializer` — wraps a delegate deserializer so a malformed *poison pill* record yields a deserialization exception routed to the container's error handler (and DLT) instead of throwing inside the poll loop and stalling the consumer. **Boot auto-config.** Spring Boot reads `spring.kafka.producer.*` / `spring.kafka.consumer.*` and creates default factories and a `KafkaTemplate`. You define explicit factory beans when you need multiple serializers, multiple container factories, transactions, or custom client property tuning. **Edge cases.** Mixing transactional and non-transactional sends needs care since transactional producers are bound per `transactional.id`. Per-record type info for `JsonDeserializer` is carried in headers by default; turning that off requires configuring a default type. Wrapping with `ErrorHandlingDeserializer` is essential in production to survive bad data.
- Why does ConsumerFactory create a new consumer per request but ProducerFactory shares one?KafkaConsumer is not thread-safe, so each concurrency thread needs its own. KafkaProducer is thread-safe and meant to be shared, so the factory caches and reuses one (plus per-transactional.id producers when transactions are on).
- How do you stop a single malformed record from killing the poll loop?Wrap the real deserializer in ErrorHandlingDeserializer (key and/or value). A failed deserialization becomes a handled exception routed to the container error handler / dead-letter topic instead of repeatedly crashing the consumer.
saying these in an interview costs you the question
- Putting deserializers on the producer factory or vice versa
- Claiming KafkaConsumer is thread-safe and can be shared across threads
- Forgetting trusted packages with JsonDeserializer (security/deserialization risk)
- Saying serializers are a KafkaTemplate concern rather than the factory's