How do Pulsar's subscription modes let one system act as both a message queue and a streaming log, and how does that compare to Kafka consumer groups?

**The problem this solves.** Historically you picked either a *message queue* (RabbitMQ/ActiveMQ: competing consumers grab individual messages, scale consumers freely, per-message ack) or a *streaming log* (Kafka: ordered partitions replayed by offset). Pulsar tries to be both through subscription modes. **Subscription basics.** A consumer subscribes to a topic using a *subscription name*. The subscription is durable server-side state tracking a *cursor* (how far that subscription has consumed). Multiple consumers can join the same subscription; the **mode** governs how messages are distributed among them: 1. **Exclusive** — only one consumer may attach; others are rejected. Strict ordering, simplest. 2. **Failover** — multiple consumers attach, but only one is *active* at a time (per partition); the rest are standbys promoted on failure. Ordered, with hot failover. 3. **Shared (round-robin)** — every consumer on the subscription gets a share of messages, distributed round-robin. This is the *competing consumers / work queue* pattern: add consumers to go faster, even beyond the partition count, with **no ordering guarantee** across the subscription and **per-message individual acknowledgement**. 4. **Key_Shared** — like Shared, but messages are routed by key hash so a given key always lands on the same consumer. You keep load balancing *and* per-key ordering. **Acknowledgement model.** Pulsar acks *individual messages* (and supports cumulative ack and *negative ack* for redelivery, plus per-message redelivery with a delay). This is what makes the queue semantics work: in Shared mode, message 5 can be acked while message 3 is still being processed by another consumer. Pulsar also has built-in **delayed/scheduled messages** and **dead-letter topics**. **Kafka's model.** Kafka has *consumer groups*. The group's members are each assigned whole *partitions*; a partition is consumed by at most one member of the group. So: - Max useful parallelism in a group = number of partitions. To add consumers you must add partitions. - Ordering is guaranteed *within a partition*, which is roughly Pulsar's Failover/Exclusive territory. - Progress is tracked with a single *committed offset* per partition, not per-message acks. You can't ack message 5 while leaving message 3 outstanding within the same partition — offset commit is a high-water mark. Re-processing a single failed message means custom retry topics/DLQ patterns (e.g. with Kafka Streams or Spring Kafka's retry/DLT), not a native per-message negative-ack. **Mapping.** Exclusive/Failover ≈ Kafka's ordered-partition consumption. Shared ≈ a real competing-consumers queue Kafka can't natively express (you'd approximate it with many partitions + custom logic). Key_Shared ≈ Kafka's key→partition routing *but* decoupled from physical partition count, so you can rebalance keys across more consumers than partitions. **Edge cases.** In Key_Shared, consumer churn can transiently block keys (Pulsar offers sticky-hash-range options to limit reshuffling). In Shared mode, ordering is genuinely absent — if you need ordering you must use Key_Shared or a single-active mode. And acking individual messages out of order in Shared mode means the cursor can have 'holes' the broker must track, which costs some memory/metadata.