skip to content

Producers

3 roadmaps36 questions

Everything on the write path: how send() batches records, what acks buys you, idempotence, partitioning, and retry semantics. Interviewers focus here because most data-loss and duplicate stories start with a producer setting.

part ofApache Kafkaoverview, primer and where to startread it →
on this pageshow

explore

questions

page 2 of 2

What is a ProducerInterceptor, and what do onSend and onAcknowledgement do, including thread and ordering semantics?

level: seniorimportance: should knowfreq 40%

basics

~10 s

A ProducerInterceptor lets you hook into the producer pipeline. onSend runs before serialization and can mutate/inspect the record; onAcknowledgement runs when the broker acks or the send fails. You configure a chain via interceptor.classes.

open as a page

How do the Protobuf and JSON Schema serializers differ from Avro, especially in wire format and reference handling?

level: seniorimportance: should knowfreq 35%

basics

~20 s

All three use the same magic-byte + schema-ID framing. Protobuf adds message-index bytes to pick the message type inside a .proto and supports schema references for imports; JSON Schema sends JSON text payloads. Compatibility is still enforced per subject.

open as a page

Design an end-to-end no-data-loss producer/topic configuration. Beyond acks=all, what settings are required and what failure modes remain?

level: principalimportance: should knowfreq 40%

basics

~20 s

Use acks=all with enable.idempotence=true on the producer, replication.factor=3 and min.insync.replicas=2 on the topic, and unclean.leader.election.enable=false on the brokers. Handle send failures (don't drop them) and bound delivery.timeout.ms. Remaining risks: simultaneous loss of all ISR replicas and consumer-side processing gaps.

open as a page

You need to maximize producer throughput for a high-volume Kafka pipeline. Which producer configs do you tune together, and what are the trade-offs and failure modes?

level: principalimportance: should knowfreq 40%

basics

~20 s

Raise batch.size and linger.ms so batches fill, enable a fast codec like lz4 or zstd via compression.type, and increase buffer.memory so the producer doesn't block. Accept added per-record latency and watch for buffer exhaustion, ordering, and broker size limits.

open as a page

When does an idempotent producer throw OutOfOrderSequenceException, and what does it imply about delivery guarantees?

level: principalimportance: should knowfreq 35%

basics

~20 s

It means the broker received a producer's batch with a sequence number that doesn't follow the last one it accepted, so a gap exists — usually because an earlier batch was permanently lost or its state expired. It signals the producer can no longer guarantee ordered, gap-free delivery for that session.

open as a page

How does the Sender thread drain the RecordAccumulator and use the NetworkClient, and how do max.in.flight.requests.per.connection and idempotence interact with batch ordering and retries?

level: principalimportance: should knowfreq 48%

basics

~20 s

The Sender thread polls the accumulator for ready batches, groups them by leader broker, and sends ProduceRequests via the NetworkClient — up to max.in.flight.requests.per.connection outstanding per connection. With idempotence on, the producer can keep 5 in-flight and still preserve per-partition order and dedup on retries; without it, retries can reorder unless in-flight is 1.

open as a page

showing 31–36 of 36