What is the kafka-reassign-partitions tool, and what are its three main phases?
answer
- generate / execute / verify
- topics-to-move-json + broker-list
- generate prints current = rollback
- verify removes throttles
- new replicas join ISR before old removed
basics
~10 skafka-reassign-partitions is the CLI tool that moves partition replicas between brokers. Its three phases are generate (propose a plan), execute (apply it), and verify (check progress/completion).
solid answer
~40 skafka-reassign-partitions.sh is Kafka's CLI for moving partition replicas across brokers — used to rebalance load, drain a broker, or add capacity. You run it in three phases. Generate (--generate) takes a topics-to-move JSON plus a list of target broker IDs and proposes a candidate assignment, also printing the current assignment so you can roll back. Execute (--execute) reads a reassignment JSON file and submits it; the controller starts moving replicas, adding new replicas to the ISR before removing old ones. Verify (--verify) reports each partition as 'in progress', 'completed', or 'failed', and removes any throttles that were applied. The generated plan is only a suggestion — operators usually edit it for rack-awareness or balance before executing.
go deeper
Know the three verbs (generate/execute/verify) and that it moves replicas between brokers.
Know the JSON inputs, that generate's plan is editable, and that verify clears throttles.
Explain the add-then-remove ISR safety, throttling, and when to hand-edit for rack-awareness.
Discuss when CLI reassignment is the wrong tool (continuous balancing → Cruise Control) and operational guardrails for large plans.
## What problem this solves In Kafka, every topic is split into **partitions**, and each partition has several **replicas** (copies) spread across **brokers** (the server processes). One replica is the **leader** (handles reads/writes); the others are **followers** that copy the leader's data. The set of replicas currently caught up is the **ISR** (in-sync replicas). When you add brokers, decommission a broker, or have uneven load, you need to physically move replicas between brokers. That movement is a **partition reassignment**, and the tool is `kafka-reassign-partitions.sh` (class `kafka.admin.ReassignPartitionsCommand`). ## The three phases **1. Generate (`--generate`)** — You pass `--topics-to-move-json-file` (a JSON listing topics) and `--broker-list "1,2,3"` (the target brokers). The tool prints two JSON blobs: the *current* assignment (save this — it is your rollback plan) and a *proposed* new assignment. The proposal is just round-robin-ish balancing; it is NOT guaranteed optimal or rack-aware, so operators typically hand-edit it. **2. Execute (`--execute`)** — You pass `--reassignment-json-file` with the (possibly edited) plan. The controller writes the new target replica set and begins moving data. Crucially, Kafka *adds* the new replicas and waits for them to catch up and join the ISR *before* removing the old replicas, so the partition never drops below its replication during the move (availability is preserved). You can also pass `--throttle <bytes/sec>` here to cap replication bandwidth. **3. Verify (`--verify`)** — Re-run with the same JSON to see status per partition: 'is still in progress', 'completed successfully', or 'failed'. Verify also *removes the throttle* once movement is done — forgetting to verify leaves the throttle config lingering on topics/brokers. ## Edge cases - Reassignments are **cluster-wide controller operations**; a large plan can saturate network/disk, hence throttling. - In modern Kafka you can `--cancel` an in-progress reassignment. - The tool does not decide *what* is balanced — for goal-based, continuous balancing you use Cruise Control instead.
- Why does the generate phase also print the current assignment?So you can save it as a rollback plan — if the reassignment goes badly you re-execute the original assignment to restore the prior layout.
- Why must you still run --verify even after execute appears done?Verify reports per-partition completion and, importantly, removes any replication throttles that --execute set; skipping it leaves throttle configs stuck on the topic/brokers.
saying these in an interview costs you the question
- Claiming the generated plan is optimal/rack-aware — it is only a naive suggestion meant to be edited.
- Saying old replicas are deleted first then new ones added (it is the reverse — availability is preserved).
- Forgetting that --verify clears throttles.