Apache Pulsar is a distributed messaging and streaming platform. Like Kafka, it carries high-throughput streams of events between producers and consumers; unlike Kafka's original design, it separates the serving layer from the storage layer, which changes how it scales and operates. Pulsar came out of Yahoo and is built around two components: brokers that handle client connections and message routing, and Apache BookKeeper bookies that durably store the data.

Separated compute and storage

Because brokers are stateless and storage lives in BookKeeper, you can scale the two independently. Adding broker capacity to handle more connections doesn't require rebalancing data, and growing storage doesn't touch the serving tier. A broker that fails can be replaced instantly because it holds no data - another broker simply picks up its topics. This is the main architectural difference from a partition-and-replica model where each node owns its own data.

Messaging models in one system

Pulsar supports both queuing and streaming from the same topics through its subscription types. An exclusive or failover subscription gives ordered, log-style consumption like a stream; a shared subscription fans messages out across many consumers like a work queue. Tiered storage can offload older segments to object storage such as S3, so a topic can retain effectively unlimited history cheaply.

consumer = client.subscribe(
    'persistent://tenant/ns/events',
    subscription_name='etl',
    subscription_type=SubscriptionType.Shared)

Where it fits

Pulsar suits organizations that want multi-tenancy (built-in tenants and namespaces), geo-replication across regions, and both queue and stream semantics in one platform. The tradeoff is operational complexity: running brokers, BookKeeper, and ZooKeeper is more moving parts than a single-tier system. It competes with Kafka and pairs with stream processors like Apache Flink for downstream computation.

Frequently asked questions

How does Pulsar differ from Kafka? Pulsar separates serving (brokers) from storage (BookKeeper), so the two scale independently and brokers are stateless and instantly replaceable; it also offers both queue and stream subscription models natively.

What is BookKeeper's role? It is the durable, replicated storage layer that holds Pulsar's message data, letting brokers stay stateless.

What is the main downside? More components to operate - brokers, BookKeeper bookies, and ZooKeeper - which raises the operational bar compared with simpler single-tier systems.