How does Redpanda's architecture (C++ thread-per-core, no JVM, no ZooKeeper, Raft) differ from Apache Kafka's, and what does each choice buy you?

## Baseline: how Kafka works A Kafka broker is a **JVM** process. The JVM provides garbage collection (automatic memory reclamation), which can introduce **GC pauses** — brief stop-the-world stalls that inflate tail latency (the slowest 1% or 0.01% of requests). Kafka historically used **ZooKeeper**, a separate clustered service, to store cluster metadata (which brokers exist, who leads each partition). Replication used the **ISR** (in-sync replicas) model: each partition has a leader and followers; the leader tracks which followers are caught up, and a write is acknowledged once the in-sync set has it (with `acks=all`). Newer Kafka uses **KRaft** (a Raft-based controller) to remove ZooKeeper, but data-path replication is still ISR. ## Redpanda's four big choices **1. Single C++ binary, no JVM.** Redpanda is written in C++ and ships as one executable. Removing the JVM removes GC entirely, so there are no GC-induced latency spikes and no heap/GC tuning. Memory is managed explicitly. This is the headline reason Redpanda advertises lower and flatter tail latency under load. **2. Thread-per-core (Seastar / shared-nothing).** Seastar is a C++ async framework where each CPU core runs one pinned thread (a **shard**) with its own slice of RAM and its own set of partitions. Cores communicate by explicit message passing, not shared locks. Benefits: no lock contention, almost no context switching, cache-friendly access, and the ability to saturate NVMe SSDs and fast NICs. It uses a **busy-poll** loop, so a Redpanda core tends to run hot even when idle-ish — high single-core utilization is by design, not a leak. **3. No ZooKeeper, no separate controller.** Redpanda embeds **Raft** for everything: cluster metadata and each partition's replication are Raft groups. One consensus algorithm covers control and data planes, reducing the number of distinct systems an operator must understand and run. **4. Raft for partition replication.** Instead of ISR, each partition is a Raft group: writes are committed when a **majority (quorum)** of replicas acknowledge. This gives a clean, formally-reasoned consistency model. A subtle difference from Kafka's tunable ISR: quorum replication ties acknowledgment to a majority rather than to a configurable in-sync set, which changes how you reason about minimum replica counts and availability under failures. ## What each choice buys you - No JVM → predictable low tail latency, no GC tuning. - Thread-per-core → high throughput per node, efficient hardware use, fewer nodes for the same load. - No ZooKeeper / unified Raft → fewer moving parts, simpler deploys and upgrades. ## Costs and caveats - You leave a huge ecosystem of JVM/Kafka operational tooling, dashboards, and tribal knowledge. - Busy-polling means CPU graphs look 'always busy'; capacity planning differs. - Storage is still **broker-local disk** — Redpanda does not, by itself, remove cross-AZ replication cost the way an object-storage design does (tiered storage to S3 is a separate, complementary feature). - Feature/protocol coverage can trail upstream Kafka for newer KIPs. ## Mental model Kafka optimizes for the JVM ecosystem and tunable ISR durability; Redpanda optimizes the engine for bare-metal performance and operational simplicity while keeping the Kafka protocol on the front door.