As an architect, when would you choose Apache Pulsar over Kafka, and when does Kafka's partition-centric design remain the better choice?

**Decide from requirements, not aesthetics.** Both are durable, ordered, partitioned, replicated pub/sub systems. The choice turns on which of Pulsar's architectural features you actually need, weighed against Kafka's ecosystem and operational maturity. **Lean Pulsar when:** 1. **You need queue AND stream semantics in one system.** Pulsar's subscription modes (Exclusive/Failover = ordered stream; Shared = work queue; Key_Shared = per-key-ordered fan-out) plus per-message ack, negative-ack, delayed messages, and native dead-letter topics cover use cases that would otherwise require Kafka *plus* RabbitMQ/SQS. Consolidating two systems into one is a real operational win. 2. **Native multi-tenancy.** A platform team hosting many internal/external tenants benefits from tenant→namespace→topic isolation with per-namespace quotas, auth, and retention, instead of bolting tenancy onto Kafka via naming conventions, ACLs, quotas, or one-cluster-per-tenant. 3. **Built-in geo-replication**, especially active-active, without operating MirrorMaker 2 as a separate Connect deployment. 4. **Independent compute/storage scaling and elastic ops.** Stateless brokers give near-instant failover and let you scale serving (brokers) separately from storage (bookies). Useful for spiky workloads and fast autoscaling. 5. **Very large topics / long retention as a stream store.** Segment-centric storage stripes a topic across all bookies, so topics can exceed a single node's disk and storage grows copy-free. Combined with **tiered storage** to object stores, Pulsar is attractive as a long-retention event store. 6. **High topic count** with frequent create/teardown — Pulsar handles large numbers of topics well. **Lean Kafka when:** 1. **Ecosystem and tooling dominate.** Kafka Connect's huge connector catalog, Kafka Streams and ksqlDB for stream processing, the de-facto Schema Registry, mature exactly-once **transactions**, and broad monitoring/ops tooling. Pulsar has analogues (Pulsar IO, Pulsar Functions, its own schema registry) but a smaller ecosystem and talent pool. 2. **Operational simplicity / team familiarity.** Kafka is one tier of brokers (now with KRaft removing ZooKeeper). Pulsar is brokers + bookies + a metadata store (+ optional function workers) — more components, more failure modes, more expertise required. For a small team, that overhead can outweigh the architectural benefits. 3. **Standard partitioned streaming is the whole job.** If you need ordered, partitioned event streams consumed by groups, and don't need work-queue fan-out, native tenancy, or built-in geo-repl, Kafka does it with less to operate. 4. **Hiring/community.** Far more engineers know Kafka; vendor support and managed offerings (MSK, Confluent Cloud) are ubiquitous. **Things people get wrong.** - *'Pulsar is always more scalable.'* It's more *elastic* (copy-free scaling, stateless brokers), but Kafka scales to enormous throughput too; the difference is operational elasticity and storage disaggregation, not a raw throughput ceiling. - *'Pulsar replaces RabbitMQ for free.'* Shared mode gives queue semantics, but you still operate BookKeeper and accept its complexity. - *'Kafka can't do long retention.'* With tiered storage (KIP-405) Kafka offloads cold segments to object storage, narrowing Pulsar's storage advantage — though the hot path stays broker-coupled. - *'Migration is trivial.'* Client protocols differ; Pulsar offers a Kafka-protocol handler (KoP) and Kafka has no Pulsar mode, so cross-migration is real engineering. **A pragmatic default.** For a typical org doing event streaming with existing Kafka skills and connector needs, Kafka is the lower-risk default. Choose Pulsar deliberately when two or more of its differentiators (queue+stream unification, multi-tenancy, geo-replication, compute/storage disaggregation, very large topics) are first-order requirements — that's when its added operational surface pays for itself.