Starlight for RabbitMQ

Starlight for RabbitMQ brings native RabbitMQ® protocol support to Apache Pulsar®, enabling migration of existing RabbitMQ applications and services to Pulsar without modifying the code. RabbitMQ applications can now leverage Pulsar’s powerful features, such as:

  • Consistent metadata store

    Starlight for RabbitMQ uses Apache ZooKeeper™, so existing Zookeeper configuration stores can store Starlight for RabbitMQ metadata.

  • Security and authentication

    Starlight for RabbitMQ connects to brokers that have TLS, authentication, and/or authorization enabled, because it uses the same AuthenticationService as Pulsar.

  • Clustering

    Launch multiple stateless Starlight for RabbitMQ instances simultaneously for scalability and high availability.

  • Multi-tenancy

    Starlight for RabbitMQ offers support for multi-tenancy, mapping an AMQP virtual host to a Pulsar tenant and namespace.

By integrating two popular event streaming ecosystems, Starlight for RabbitMQ unlocks new use cases and reduces barriers for users adopting Pulsar. Leverage advantages from each ecosystem and build a truly unified event streaming platform with Starlight for RabbitMQ to accelerate the development of real-time applications and services.

Get started producing and consuming RabbitMQ messages on a Pulsar cluster.

Starlight for RabbitMQ Quickstart

  1. To start connecting Starlight for RabbitMQ, select RabbitMQ in the Astra Streaming Connect tab.

  2. When the popup appears, confirm you want to enable RabbitMQ on your tenant.

    You will not be able to remove the RabbitMQ namespace created on your tenant with this step.

  3. Select Enable RabbitMQ.

    A new rabbitmq namespace is created in your Astra Streaming tenant for RabbitMQ functionality.

    A new configuration file is generated in the Connect tab that looks like this:

    username:
    password: token:***
    host: rabbitmq-azure-us-west-2.dev.streaming.datastax.com
    port: 5671
    virtual_host: azure/rabbitmq
  4. Copy and paste the code or download it as a configuration file (it will be called rabbitmq.conf).

You’re now ready to connect RabbitMQ and Pulsar.

Test connection

We’ll use a Python script to create a connection between RabbitMQ and your Pulsar tenant, establish a message queue named queuename, print ten messages, and close the connection.

  1. Create a file called connect-test.py and paste the below Python code into it:

    import ssl
    import pika
    
    virtual_host = "{your_tenant_name}/rabbitmq"
    token = "{pulsar_token}"
    
    context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
    context.verify_mode = ssl.CERT_NONE
    context.check_hostname = False
    context.load_default_certs()
    ssl_options = pika.SSLOptions(context)
    
    connection = pika.BlockingConnection(pika.ConnectionParameters(
        virtual_host=virtual_host,
        host="{your_host_name}",
        ssl_options=ssl_options,
        port=5671,
        credentials=pika.PlainCredentials("", token)))
    print("connection success")
    
    channel = connection.channel()
    print("started a channel")
    
    channel.queue_declare(queue='queuename')
    
    for x in range(10):
        channel.basic_publish(exchange='',
                          routing_key='routingkey',
                          body='message body goes here')
        print(" sent one")
    
    connection.close()
  2. Replace the following values in connect-test.py with values from your downloaded rabbitmq.conf file:

    • virtual_host

    • token

    • host

    • port

  3. Save connect-test.py with the new values.

  4. Run connect-test.py with python3 connect-test.py. It should return:

    python3 connect-test.py
    connection success
    started a channel
     sent one
     sent one
     sent one
     sent one
     sent one
     sent one
     sent one
     sent one
     sent one
     sent one
  5. Navigate to your rabbitmq namespace dashboard in Astra Streaming and monitor your activity.

You should see new topics called amq.default.__queuename and amq.default_routingkey that were created by the Python script above, and an increasing amount of traffic and messages. Your RabbitMQ messages are being published to a Pulsar topic.

RabbitMQ exchanges and Pulsar topics

Starlight for RabbitMQ maps the RabbitMQ concept of exchanges to the Pulsar concept of topics. This table shows how those concepts are mapped and used:

Exchange Routing key Pulsar topic name Code example

amp.direct

used

amq.direct.__{routing key}

channel.basic_publish(exchange='amp.direct',

amp.default or empty string

used

amq.default.__{routing key}

channel.basic_publish(exchange=",

amp.match

not used

amp.match

channel.basic_publish(exchange=amp.match,

amp.fanout

not used

amp.fanout

channel.basic_publish(exchange='amp.fanout',

headers

not used

Name of the header

channel.exchange_declare(exchange='header_logs', exchange_type='headers') channel.basic_publish(exchange='header_logs',

What’s Next?

Starlight for RabbitMQ is based on the DataStax Starlight for RabbitMQ project.
For more on Astra Streaming, see Astra Streaming quickstart.
For using Kafka with Astra Streaming, see Starlight for Kafka.