How are transaction markers timestamped, and what timestamp do committed records carry under LogAppendTime vs CreateTime?

## Timestamp types Every Kafka record has a timestamp. The topic config **`message.timestamp.type`** controls its meaning: - **CreateTime** (default): the timestamp is whatever the *producer* set (usually `System.currentTimeMillis()` at send, or an explicit value via `ProducerRecord(timestamp, ...)`). The broker preserves it. - **LogAppendTime**: the *partition leader* overwrites the timestamp with its own wall clock at the instant it appends the batch to the log. Every record in that appended batch effectively reports the broker append time. The producer's original timestamp is discarded for storage purposes. The choice affects: log **retention** (`log.retention.ms` compares against record timestamps), the **time index** (`.timeindex`, used by `offsetsForTimes`), and any **event-time** processing in Kafka Streams or consumers. ## How markers fit in A **transaction marker** is a control batch appended by the partition leader when it handles the coordinator's `WriteTxnMarkers` request. Like any append, the leader assigns it a timestamp using the broker clock. Markers are control batches, so: - They are **not** application records; the consumer never delivers them and they do not appear in user-facing time-based queries the way data does. - The marker's own timestamp is essentially the *coordinator-driven append time* — i.e., the time the transaction was finalized on that partition, not the time the data was produced. ## The key correctness point: commit does not rewrite data timestamps The data records of a transaction are appended **when the producer sends them** (during the open transaction), long before the marker is written at commit. Their timestamps are fixed at that first append: - Under **CreateTime**, they keep the producer's timestamps permanently. - Under **LogAppendTime**, they were stamped with the broker append time *at the moment of the original data append*, NOT at marker/commit time. So if a transaction stays open for 30 seconds, its records still carry timestamps from when they were produced/appended, even though the COMMIT marker (and thus visibility) happens 30 seconds later. This matters for stream processing: event-time windows key off the record timestamps, not the commit/marker time, so a long transaction can produce records whose event-time is well before they become visible — a source of apparent 'late' data and watermark complications in read_committed pipelines. ## Edge cases / propagation - The batch **maxTimestamp** and time-index entries are derived from the data batches; control (marker) batches are handled specially so they don't corrupt application time indexing. - Because LogAppendTime is set at original append, replaying/compaction does not change it; markers do not 'propagate' a new timestamp onto the data. - Mixing producers with skewed clocks under CreateTime can make retention/ordering surprising; LogAppendTime trades producer event-time fidelity for monotone broker-side time. Transactions don't alter this trade-off — only original append time matters.