What changed between the original exactly_once and exactly_once_v2 (KIP-447), and why was v2 introduced?

## Background: how the original EOS fenced zombies Kafka transactions identify a logical producer by a stable **transactional.id**. The broker's transaction coordinator tracks a **producer epoch** per transactional.id; when a new producer with the same id calls `initTransactions()`, the epoch bumps and any older producer (a ‘zombie’) is **fenced** (its sends/commits fail with `ProducerFencedException`). To make this fencing correct across rebalances, the original Streams EOS gave **each task a unique transactional.id** derived from the application id + task id (partition). Reason: a partition can move to another instance during a rebalance, and the only way to guarantee the old owner can't still commit was to tie the transactional.id to the partition, so whoever owns the partition next bumps the epoch and fences the previous owner. ## Why that didn't scale One transactional.id per partition means **one transactional producer per partition** (a producer can run only one transactional.id). Consequences: - N partitions → N producers per instance → N× broker connections, memory, threads. - Each rebalance reassigns partitions, so producers must be **closed and recreated** and re-run `initTransactions()` — slow, and it touches the transaction coordinator a lot. - Transaction state on the coordinator grows with the number of transactional.ids. This made EOS painful at high partition counts and made rebalances a latency cliff. ## What KIP-447 changed KIP-447 decouples zombie fencing from per-partition transactional.ids. The insight: the **consumer group** already coordinates partition ownership and has a generation/epoch. By passing **`ConsumerGroupMetadata`** (group id, generation id, member id, instance id) into `producer.sendOffsetsToTransaction(...)`, the transaction coordinator can ask the **group coordinator** whether this producer's consumer is still the current owner. A producer whose consumer has been rebalanced out (stale generation) is fenced when it tries to commit offsets — without needing a partition-scoped transactional.id. Result: Streams uses **one transactional.id per StreamThread** (thread-level), so **one producer per thread**, regardless of how many partitions/tasks that thread handles. Far fewer producers, far cheaper rebalances. ## Naming / version timeline - 2.5: shipped as `exactly_once_beta`. - 2.6: renamed `exactly_once_v2`; `exactly_once_beta` deprecated as an alias. - 3.0: original `exactly_once` deprecated; v2 is the recommended value. - Requirement: brokers and the transaction/group coordinator must be **>= 2.5** because the broker side must support the metadata-based fencing API. ## Migration notes - Upgrade brokers to >= 2.5 first. - Because the transactional.id scheme changes, an in-place rolling upgrade from `exactly_once` to `exactly_once_v2` must follow the documented two-rolling-bounce procedure to avoid mixing fencing models; otherwise zombie fencing could be momentarily incorrect. ## Edge cases - v2 still requires single-cluster transactions and durable output config. - Throughput improves but EOS still adds commit-marker and read_committed latency. - Per-thread producer means a single producer error can affect all tasks on that thread (broader blast radius than per-task producers) — a deliberate tradeoff for scalability.