Shared subscriptions in Pulsar

Subscriptions in Pulsar describe which consumers are consuming data from a topic and how they want to consume that data.

A shared subscription allows multiple consumers to consume messages from a single topic in round-robin fashion.
More consumers in a shared subscription can increase your Pulsar deployment’s rate of message consumption, but at the cost of losing message ordering guarantees and acknowledgement schemes.

This document explains how to use Pulsar’s shared subscription model to manage your topic consumption.

Prerequisites

To run this example, you’ll need:

Apache Maven
Java OpenJDK 11
A configured Astra Streaming instance with at least one streaming tenant and one topic. See the Astra Streaming quick start for instructions.
A local clone of the DataStax Pulsar Subscription Example repository
Modify the src/main/resources/application.properties in the pulsar-subscription-example repo to connect to your Astra Streaming cluster, as below:
```
service_url={broker-service-url}
namespace=default
tenant_name=my-tenant
authentication_token={astra-auth-token}
topic_name=my-topic
```

Shared subscription example

To try out a Pulsar shared subscription, add .subscriptionType(SubscriptionType.Shared) to the pulsarConsumer in SimplePulsarConsumer.java.

pulsarConsumer = pulsarClient.newConsumer(Schema.JSON(DemoBean.class))
    .topic("persistent://"
        + conf.getTenantName() + "/"
        + conf.getNamespace() + "/"
        + conf.getTopicName())
    .startMessageIdInclusive()
    .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
    .subscriptionName("SimplePulsarConsumer")
    .subscriptionType(SubscriptionType.Shared)
    .subscribe();

Open pulsar-subscription-example in the IDE of your choice and run SimplePulsarConsumer.java to begin consuming messages.
The confirmation message and a cursor appear to indicate the consumer is ready.

[main] INFO com.datastax.pulsar.Configuration - Configuration has been loaded successfully
...
[pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://<tenant_name>/<namespace>/in][SimplePulsarConsumer] Subscribed to topic on <service_url> -- consumer: 0

In a new terminal window, run SimplePulsarProducer.java to begin producing messages.

[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 59819331 sent
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 70129519 sent
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 31365142 sent
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 48206643 sent
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 51277375 sent

The consumer begins receiving messages.

[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":59819331,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":31365142,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":51277375,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}

Open a new terminal window and run SimplePulsarConsumer.java. The new consumer subscribes to the topic and consumes a message.

[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":70129519,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":48206643,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}

Open as many terminals as you’d like and run SimplePulsarConsumer.java. All the consumers subscribe to the topic and consume messages in a round-robin fashion.

If you run this test with exclusive subscriptions, you can’t attach more than once subscriber to the exclusive topic. Since this test uses shared subscriptions, you can attach multiple consumers to the topic.

Shared subscription video

Follow along with this video from our Five Minutes About Pulsar series to see shared subscriptions in action.

What’s next?

For more on subscriptions, see:

Shared subscriptions in Pulsar

Prerequisites

Shared subscription example

Shared subscription video

What’s next?

Was this helpful?

Give Feedback