skip to content

What is the kafka-reassign-partitions tool, and what are its three main phases?

level: juniorimportance: must knowfreq 70%

answer

  1. generate / execute / verify
  2. topics-to-move-json + broker-list
  3. generate prints current = rollback
  4. verify removes throttles
  5. new replicas join ISR before old removed

basics

~10 s

kafka-reassign-partitions is the CLI tool that moves partition replicas between brokers. Its three phases are generate (propose a plan), execute (apply it), and verify (check progress/completion).

solid answer

~40 s

kafka-reassign-partitions.sh is Kafka's CLI for moving partition replicas across brokers — used to rebalance load, drain a broker, or add capacity. You run it in three phases. Generate (--generate) takes a topics-to-move JSON plus a list of target broker IDs and proposes a candidate assignment, also printing the current assignment so you can roll back. Execute (--execute) reads a reassignment JSON file and submits it; the controller starts moving replicas, adding new replicas to the ISR before removing old ones. Verify (--verify) reports each partition as 'in progress', 'completed', or 'failed', and removes any throttles that were applied. The generated plan is only a suggestion — operators usually edit it for rack-awareness or balance before executing.

go deeper

for a junior

Know the three verbs (generate/execute/verify) and that it moves replicas between brokers.

for a middle

Know the JSON inputs, that generate's plan is editable, and that verify clears throttles.

for a senior

Explain the add-then-remove ISR safety, throttling, and when to hand-edit for rack-awareness.

for a principal

Discuss when CLI reassignment is the wrong tool (continuous balancing → Cruise Control) and operational guardrails for large plans.

## What problem this solves In Kafka, every topic is split into **partitions**, and each partition has several **replicas** (copies) spread across **brokers** (the server processes). One replica is the **leader** (handles reads/writes); the others are **followers** that copy the leader's data. The set of replicas currently caught up is the **ISR** (in-sync replicas). When you add brokers, decommission a broker, or have uneven load, you need to physically move replicas between brokers. That movement is a **partition reassignment**, and the tool is `kafka-reassign-partitions.sh` (class `kafka.admin.ReassignPartitionsCommand`). ## The three phases **1. Generate (`--generate`)** — You pass `--topics-to-move-json-file` (a JSON listing topics) and `--broker-list "1,2,3"` (the target brokers). The tool prints two JSON blobs: the *current* assignment (save this — it is your rollback plan) and a *proposed* new assignment. The proposal is just round-robin-ish balancing; it is NOT guaranteed optimal or rack-aware, so operators typically hand-edit it. **2. Execute (`--execute`)** — You pass `--reassignment-json-file` with the (possibly edited) plan. The controller writes the new target replica set and begins moving data. Crucially, Kafka *adds* the new replicas and waits for them to catch up and join the ISR *before* removing the old replicas, so the partition never drops below its replication during the move (availability is preserved). You can also pass `--throttle <bytes/sec>` here to cap replication bandwidth. **3. Verify (`--verify`)** — Re-run with the same JSON to see status per partition: 'is still in progress', 'completed successfully', or 'failed'. Verify also *removes the throttle* once movement is done — forgetting to verify leaves the throttle config lingering on topics/brokers. ## Edge cases - Reassignments are **cluster-wide controller operations**; a large plan can saturate network/disk, hence throttling. - In modern Kafka you can `--cancel` an in-progress reassignment. - The tool does not decide *what* is balanced — for goal-based, continuous balancing you use Cruise Control instead.

  • Why does the generate phase also print the current assignment?
    So you can save it as a rollback plan — if the reassignment goes badly you re-execute the original assignment to restore the prior layout.
  • Why must you still run --verify even after execute appears done?
    Verify reports per-partition completion and, importantly, removes any replication throttles that --execute set; skipping it leaves throttle configs stuck on the topic/brokers.

saying these in an interview costs you the question

  • Claiming the generated plan is optimal/rack-aware — it is only a naive suggestion meant to be edited.
  • Saying old replicas are deleted first then new ones added (it is the reverse — availability is preserved).
  • Forgetting that --verify clears throttles.

context

resolving links…