How does a follower replica actually stay in sync with the leader? Describe the ReplicaFetcherThread and its fetch loop.

**The pull model.** A crucial design choice: Kafka replication is **pull-based**, not push-based. The leader never proactively sends data to followers. Instead each follower *asks* the leader for data using the very same **Fetch** API that consumers use. This means the leader is largely stateless about replication progress — it learns each follower's position from the offsets in their fetch requests. **Who runs the loop.** On a broker acting as a follower, the `ReplicaManager` owns a `ReplicaFetcherManager`. The manager creates a pool of **`ReplicaFetcherThread`** objects. Partitions are assigned to threads by a hash of the partition over the configured fetcher count, *grouped by the leader broker* — i.e. one thread handles partitions whose leader is broker X, another handles partitions led by broker Y. So the number of fetcher threads a follower runs toward one leader is bounded by `num.replica.fetchers` (default 1). **The loop, step by step.** 1. The thread assembles a Fetch request. For each assigned partition it includes `fetchOffset = follower's current log-end offset (LEO)` — the offset of the next record it needs. 2. It sends the request to the leader broker and waits. The leader uses *long polling*: if there is no new data, it holds the request open up to **`replica.fetch.wait.max.ms`** (default 500 ms) or until at least `replica.fetch.min.bytes` (default 1) of data accumulates, then responds. 3. The leader returns records starting at `fetchOffset`, capped at **`replica.fetch.max.bytes`** (default ~1 MB) per partition and `replica.fetch.response.max.bytes` for the whole response. 4. The follower **appends** the returned records to its local log segment, advancing its LEO. 5. The arrival of the *next* fetch request — now carrying the higher `fetchOffset` — tells the leader the follower has persisted up to that point. The leader records this as the follower's LEO and uses the minimum LEO across the ISR to advance the **high watermark** (the highest offset considered committed/readable by consumers). **Why offsets-as-acknowledgment is elegant.** The follower's progress is implicit in its next request's start offset. No separate ack protocol is needed; the fetch request *is* the acknowledgment of everything below its `fetchOffset`. **Edge cases & nuances.** - **Catch-up vs steady state:** A lagging follower's fetches return full batches every time (bounded by the byte limits); a caught-up follower's fetches mostly block and return empty. - **Offset out of range / truncation:** If a follower's offset doesn't match the leader's log (e.g. after an unclean failover), the follower truncates its log to the leader's, using leader epochs (KIP-101/KIP-279) to find the correct divergence point, then resumes fetching. - **Throttling:** Replication can be rate-limited (`leader.replication.throttled.rate` / `follower...`), used during reassignment so fetchers don't saturate the network. - **One thread, many partitions:** Because a single thread multiplexes many partitions, a slow/large partition can delay others on the same thread — a reason to raise `num.replica.fetchers`.