Starlight for RabbitMQ
Starlight for RabbitMQ brings native RabbitMQ® protocol support to Apache Pulsar®, enabling migration of existing RabbitMQ applications and services to Pulsar without modifying the code. RabbitMQ applications can now leverage Pulsar’s powerful features, such as:
-
Consistent metadata store
Starlight for RabbitMQ uses Apache ZooKeeper™, so existing Zookeeper configuration stores can store Starlight for RabbitMQ metadata.
-
Security and authentication
Starlight for RabbitMQ connects to brokers that have TLS, authentication, and/or authorization enabled, because it uses the same
AuthenticationService
as Pulsar. -
Clustering
Launch multiple stateless Starlight for RabbitMQ instances simultaneously for scalability and high availability.
-
Multi-tenancy
Starlight for RabbitMQ offers support for multi-tenancy, mapping an AMQP virtual host to a Pulsar tenant and namespace.
By integrating two popular event streaming ecosystems, Starlight for RabbitMQ unlocks new use cases and reduces barriers for users adopting Pulsar. Leverage advantages from each ecosystem and build a truly unified event streaming platform with Starlight for RabbitMQ to accelerate the development of real-time applications and services.
Get started producing and consuming RabbitMQ messages on a Pulsar cluster.
Starlight for RabbitMQ Quickstart
-
To start connecting Starlight for RabbitMQ, select RabbitMQ in the Astra Streaming Connect tab.
-
When the popup appears, confirm you want to enable RabbitMQ on your tenant.
You will not be able to remove the RabbitMQ namespace created on your tenant with this step.
-
Select Enable RabbitMQ.
A new
rabbitmq
namespace is created in your Astra Streaming tenant for RabbitMQ functionality.A new configuration file is generated in the Connect tab that looks like this:
username: password: token:*** host: rabbitmq-azure-us-west-2.dev.streaming.datastax.com port: 5671 virtual_host: azure/rabbitmq
-
Copy and paste the code or download it as a configuration file (it will be called
rabbitmq.conf
).
You’re now ready to connect RabbitMQ and Pulsar.
Test connection
We’ll use a Python script to create a connection between RabbitMQ and your Pulsar tenant, establish a message queue named queuename
, print ten messages, and close the connection.
-
Create a file called
connect-test.py
and paste the below Python code into it:import ssl import pika virtual_host = "{your_tenant_name}/rabbitmq" token = "{pulsar_token}" context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2) context.verify_mode = ssl.CERT_NONE context.check_hostname = False context.load_default_certs() ssl_options = pika.SSLOptions(context) connection = pika.BlockingConnection(pika.ConnectionParameters( virtual_host=virtual_host, host="{your_host_name}", ssl_options=ssl_options, port=5671, credentials=pika.PlainCredentials("", token))) print("connection success") channel = connection.channel() print("started a channel") channel.queue_declare(queue='queuename') for x in range(10): channel.basic_publish(exchange='', routing_key='routingkey', body='message body goes here') print(" sent one") connection.close()
-
Replace the following values in
connect-test.py
with values from your downloadedrabbitmq.conf
file:-
virtual_host
-
token
-
host
-
port
-
-
Save
connect-test.py
with the new values. -
Run
connect-test.py
withpython3 connect-test.py
. It should return:python3 connect-test.py connection success started a channel sent one sent one sent one sent one sent one sent one sent one sent one sent one sent one
-
Navigate to your
rabbitmq
namespace dashboard in Astra Streaming and monitor your activity.
You should see new topics called amq.default.__queuename
and amq.default_routingkey
that were created by the Python script above, and an increasing amount of traffic and messages. Your RabbitMQ messages are being published to a Pulsar topic.
RabbitMQ exchanges and Pulsar topics
Starlight for RabbitMQ maps the RabbitMQ concept of exchanges to the Pulsar concept of topics. This table shows how those concepts are mapped and used:
Exchange | Routing key | Pulsar topic name | Code example |
---|---|---|---|
amp.direct |
used |
amq.direct.__{routing key} |
channel.basic_publish(exchange='amp.direct', |
amp.default or empty string |
used |
amq.default.__{routing key} |
channel.basic_publish(exchange=", |
amp.match |
not used |
amp.match |
channel.basic_publish(exchange=amp.match, |
amp.fanout |
not used |
amp.fanout |
channel.basic_publish(exchange='amp.fanout', |
headers |
not used |
Name of the header |
channel.exchange_declare(exchange='header_logs', exchange_type='headers') channel.basic_publish(exchange='header_logs', |
What’s Next?
-
Starlight for RabbitMQ is based on the DataStax Starlight for RabbitMQ project.
-
Follow our simple guide to get started with Astra now.
-
For using Kafka with Astra Streaming, see Starlight for Kafka.