What are MockProducer and MockConsumer, and how would you use them to unit-test producer and consumer logic?
answer
- in-memory Producer/Consumer impls, no broker
- MockProducer.history() asserts sends
- autoComplete true vs completeNext()/errorNext()
- MockConsumer addRecord + poll(); committed() offsets
- no real serde/partitioner -> not integration
basics
~20 sThey are in-memory fakes of the Kafka client. MockProducer records every send() in a history() list so you assert what was sent; MockConsumer lets you preload records and control poll() returns to test consumer logic — both with no broker.
solid answer
~40 sMockProducer<K,V> and MockConsumer<K,V> (in org.apache.kafka.clients) are official test doubles that implement the Producer/Consumer interfaces without a broker. MockProducer captures each send() into history() (a list of ProducerRecords); with autoComplete=true it completes futures immediately, or with autoComplete=false you call completeNext()/errorNext() to drive callbacks and test retry/error paths. You assert your code produced the right records to the right topics/partitions. MockConsumer is seeded with assignment/partitions, then you addRecord(...) and the next poll() returns them; you can also fire rebalanceListener callbacks and updateBeginningOffsets/updateEndOffsets to test seek logic, and inspect committed offsets via committed(...). These are pure unit-test tools: fast, deterministic, no serialization over the wire. Their limit is exactly that — they skip real serde, partitioning by the actual partitioner, and broker coordination, so they don't replace integration tests.
go deeper
Knows MockProducer/MockConsumer are broker-free fakes for unit-testing client logic.
Uses history(), addRecord/poll, and autoComplete modes to assert sends, consumption, and error paths.
Knows their boundary vs integration tests and when Spring abstractions make @EmbeddedKafka the better choice.
Guides when raw-client mocking is appropriate vs Spring integration, keeping the unit/integration split clean across the codebase.
Kafka's Java client exposes `Producer<K,V>` and `Consumer<K,V>` interfaces; the real implementations (`KafkaProducer`/`KafkaConsumer`) talk to a broker. For unit tests, Kafka ships in-memory implementations of those same interfaces so your code can be tested with no broker. **MockProducer** (`org.apache.kafka.clients.producer.MockProducer`): - Every `send(record)` is appended to an in-memory list returned by `history()`. You assert: 'my service sent a record to topic X with key K and these headers.' - **autoComplete mode**: constructed with `autoComplete=true`, each `send()` future completes immediately with a fake `RecordMetadata` — good for happy-path tests. - **manual mode**: with `autoComplete=false`, the futures stay pending until you call `completeNext()` (success) or `errorNext(exception)` (failure). This lets you test asynchronous callbacks, error handling, and retry logic deterministically. - It can be given a `Partitioner` and cluster metadata if you want partition assignment to be computed; otherwise it records what you sent. - Transactions are partially modeled: `beginTransaction()`, `commitTransaction()`, `abortTransaction()` are tracked so you can assert transactional flow. **MockConsumer** (`org.apache.kafka.clients.consumer.MockConsumer`): - Constructed with an `OffsetResetStrategy` (e.g. `EARLIEST`). You assign partitions with `assign(...)` or simulate subscription + `rebalance(...)`. - You seed data with `addRecord(consumerRecord)`; the next `poll(timeout)` returns the buffered records, so you can feed your consumer loop deterministic input. - You set position bounds with `updateBeginningOffsets(...)` / `updateEndOffsets(...)` to test seek-to-beginning/end logic. - Committed offsets are inspectable via `committed(...)`, and you can invoke a configured `ConsumerRebalanceListener` to test `onPartitionsRevoked/Assigned` handling. **Why use them**: they make consumer/producer-side business logic testable in milliseconds, with full control over success/failure/timing, and zero external infrastructure. Classic uses: 'does my outbox publisher send the right event?', 'does my consumer commit only after successful processing?', 'does my retry kick in when the send fails?'. **Their boundary** (where they intentionally stop): they do not perform real serialization across a broker, do not run the real partitioner/coordinator, and do not exercise rebalancing across real members. A serde mismatch, a partitioning bug against the real partitioner, or a rebalance race will NOT be caught here — those belong in `@EmbeddedKafka`/Testcontainers integration tests. Use the mocks for logic, the broker for wiring. **Note on Spring**: Spring's `KafkaTemplate` and `@KafkaListener` are usually integration-tested with `@EmbeddedKafka` instead of these raw mocks, because the Spring abstractions add their own machinery; MockProducer/MockConsumer shine when you use the plain Kafka client directly.
- How do you test a producer's error/retry path with MockProducer?Construct it with autoComplete=false so send() futures stay pending, then call errorNext(exception) to fail the send and assert your retry/error-handling logic runs; completeNext() drives the success path.
- Why wouldn't you use MockConsumer to verify rebalancing behavior?MockConsumer simulates rebalance callbacks but doesn't run the real group coordinator or multiple members; true rebalancing semantics need a real broker via @EmbeddedKafka or Testcontainers.
saying these in an interview costs you the question
- Saying MockProducer serializes records through the configured serializer against a broker — it stores them in memory.
- Using MockConsumer to claim you've tested real consumer-group rebalancing.
- Forgetting autoComplete=false is what enables testing async failure/retry with errorNext().
- Reaching for raw mocks to test Spring KafkaTemplate/@KafkaListener wiring (prefer @EmbeddedKafka there).