In a KafkaShareConsumer, how do implicit and explicit acknowledgement differ, and how does share.acknowledgement.mode change the poll loop?
answer
- default is implicit
- next poll accepts the batch
- explicit: every record, every poll
- IllegalStateException both ways
basics
~10 sWith share.acknowledgement.mode=implicit, the default, the next poll() or commit accepts the whole previous batch; with explicit, the application calls acknowledge(record, type) for every record before polling again, which allows per-record RELEASE, REJECT and RENEW.
solid answer
~30 s`share.acknowledgement.mode` defaults to `implicit`: you never call `acknowledge()` (it throws `IllegalStateException`), and the next `poll()` — or `commitSync()`/`commitAsync()` — marks every record from the previous poll as `ACCEPT`. `close()` releases the outstanding records instead, so commit first. With `explicit`, you call `acknowledge(record, AcknowledgeType.X)` on every record before the next `poll()`, or that `poll()` throws `IllegalStateException`. The calls are local; they reach the broker on the next `poll()`, a commit, or `close()`. Implicit fits batches that succeed or fail together; explicit is needed once a record can be released, rejected or renewed on its own.
code
java · 25 linesProperties props = new Properties();
props.setProperty("bootstrap.servers", "localhost:9092");
props.setProperty("group.id", "thumbnails");
props.setProperty("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.setProperty("value.deserializer", "org.apache.kafka.common.serialization.ByteArrayDeserializer");
props.setProperty("share.acknowledgement.mode", "explicit");
try (KafkaShareConsumer<String, byte[]> consumer = new KafkaShareConsumer<>(props)) {
consumer.subscribe(List.of("image-jobs"));
while (running) {
ConsumerRecords<String, byte[]> records = consumer.poll(Duration.ofMillis(1000));
for (ConsumerRecord<String, byte[]> record : records) {
try {
makeThumbnail(record.value());
consumer.acknowledge(record, AcknowledgeType.ACCEPT);
} catch (CorruptImageException e) {
consumer.acknowledge(record, AcknowledgeType.REJECT);
} catch (Exception e) {
consumer.acknowledge(record, AcknowledgeType.RELEASE);
}
}
Map<TopicIdPartition, Optional<KafkaException>> result = consumer.commitSync();
result.forEach((tp, error) -> error.ifPresent(e -> log.warn("ack failed for {}", tp, e)));
}
}go deeper
Know the config name, that implicit is the default, and that explicit means calling acknowledge() on each record.
Explain when acknowledgements are actually sent, what poll() and close() do in each mode, and which IllegalStateException each mode throws.
Spot the silent-accept bug in an implicit loop that swallows exceptions, and wire commitSync results or a commit callback into error handling.
Decide a team default: implicit for all-or-nothing batch jobs, explicit wherever records fail individually, enforced in a shared consumer template.
## Two ways to tell the broker a record is done A `KafkaShareConsumer` is the Java client for a Kafka **share group**, a group whose members share partitions and acknowledge records one by one. Every record it receives from `poll()` is held under an acquisition lock until the consumer acknowledges it. The client config `share.acknowledgement.mode` chooses how that happens, and its default is `implicit`. ## Implicit acknowledgement (the default) In implicit mode the application never calls `acknowledge()`; in fact `acknowledge()` throws `IllegalStateException` ("Implicit acknowledgement of delivery is being used"). Acknowledgement happens as a side effect of the next call: - **The next `poll()`** marks every record delivered by the previous poll as `ACCEPT` and commits those acknowledgements asynchronously. A failure to commit them raises no exception. - **`commitSync()` or `commitAsync()`** marks all delivered records as `ACCEPT` and commits them. This costs one extra request but lets you see the outcome; `commitSync()` returns a `Map<TopicIdPartition, Optional<KafkaException>>`. - **`close()`** does not accept the outstanding batch: it releases any acquired records without acknowledgement, so they are delivered again. Commit before closing if the last batch was processed. Implicit mode is all-or-nothing per batch. If processing one record throws and the loop simply carries on to the next `poll()`, that failed record is accepted with the rest. To get it retried, the batch must not be accepted — the process dies or closes without committing, so the whole batch is redelivered — or the consumer switches to explicit mode. ## Explicit acknowledgement With `share.acknowledgement.mode=explicit` the application must acknowledge **every** record returned by `poll()` with `acknowledge(record, type)` — `ACCEPT`, `RELEASE`, `REJECT` or `RENEW` — before it polls again. Calling `poll()` with records still unacknowledged throws `IllegalStateException`; the remaining records can still be acknowledged afterwards. 1. `acknowledge()` only updates local state in the consumer; nothing is sent yet. 2. The acknowledgements reach the broker on the next `poll()` (asynchronously, no exception on failure), on `commitSync()` / `commitAsync()`, or on `close()`, which tries to commit pending acknowledgements and releases whatever is still acquired. 3. For each topic-partition, the acknowledgements of a batch are applied atomically, so there is one error per partition to handle. 4. An `AcknowledgementCommitCallback`, registered with `setAcknowledgementCommitCallback()`, reports the outcome of asynchronous commits. ## What the broker sees - Both modes end with the same broker-side outcomes per record: acknowledged, available again, archived, or still acquired under a renewed lock. - Implicit mode can only express `ACCEPT`; records come back to the group only when a lock expires or `close()` releases them. Explicit mode can express all four types. - Acknowledgements committed by `poll()` travel on the share-fetch request itself. `commitSync()` and `commitAsync()` send an additional request, which the client documentation calls slightly less efficient, in exchange for a visible result. ## How to choose | Situation | Mode | |---|---| | Every record either succeeds or the process dies | implicit | | Some records fail permanently and should not be retried | explicit, `REJECT` | | Transient failures should be retried by any member | explicit, `RELEASE` | | Processing can outlast the lock | explicit, `RENEW` | | You need per-record failure handling at all | explicit | The rule of thumb: implicit is the shortest loop and is correct when a batch is effectively all-or-nothing; explicit is required as soon as one record's outcome can differ from its neighbours'. ## Things that trip people up - **Mixing the APIs.** A share consumer has `subscribe()`, `poll()`, `acknowledge()`, `commitSync()` and `commitAsync()`, but no `seek()`, `assign()` or offset-taking commit. Code ported from `KafkaConsumer` that calls those methods does not compile. - **Copying consumer-group configs.** The share consumer rejects `enable.auto.commit`, `auto.offset.reset`, `isolation.level`, `group.instance.id`, `session.timeout.ms` and `heartbeat.interval.ms` with a `ConfigException`; the equivalents that exist are group configs on the broker side. - **The mode is fixed per consumer instance.** It is read once from the config when the consumer is built. - **Batch size.** In the default `share.acquire.mode=batch_optimized` a poll may return more than `max.poll.records`, aligned to stored batches; `record_limit` caps it at `max.poll.records` and stops prefetching, which matters when each record is slow.
- What happens in implicit mode when processing one record of a batch throws and the loop continues?The next `poll()` accepts the whole previous batch, including the record that failed, so its failure is lost. Implicit mode has no way to mark one record of a batch as failed. Either let the process die so the locks expire and the batch is redelivered, or switch to explicit mode and `RELEASE` or `REJECT` that record.
- How do you find out that an acknowledgement commit failed in a share consumer?`commitSync()` returns a map from each `TopicIdPartition` to an optional `KafkaException`, one result per partition because a batch's acknowledgements are applied atomically per partition. For asynchronous commits — `commitAsync()` or the commit piggybacked on `poll()` — register an `AcknowledgementCommitCallback` with `setAcknowledgementCommitCallback()`; the piggybacked commit raises no exception itself.
- In implicit mode you still call acknowledge() for failed records.
- Explicit mode sends each acknowledge() call to the broker immediately.
- Unacknowledged records in explicit mode are accepted on the next poll().
- close() in implicit mode accepts the last batch it received.
- A share consumer commits offsets with commitSync(offsets) like KafkaConsumer.