Using Starlight for RabbitMQ with Luna Streaming
Starlight for RabbitMQ brings native RabbitMQ protocol support to Apache Pulsar™ by introducing a RabbitMQ protocol handler on Pulsar brokers or Pulsar proxies. By adding the Starlight for RabbitMQ protocol handler to your Pulsar cluster, you can migrate your existing RabbitMQ applications and services to Pulsar without modifying the code.
Prerequisites
You will need the following prerequisites in place to complete this guide:
-
Helm 3 CLI (we used version 3.8.0)
-
Kubectl CLI (we used version 1.23.4)
-
Python (we used version 3.8.10)
-
Enough access to a K8s cluster to create a namespace, deployments, and pods
Install Luna Streaming helm chart
-
Add the DataStax Helm chart repo to your Helm store.
helm repo add datastax-pulsar https://datastax.github.io/pulsar-helm-chart -
Install the Helm chart using a minimalist values file.
This command creates a Helm release named "my-pulsar-cluster" using the DataStax Luna Helm chart, within the K8s namespace "datastax-pulsar". The minimal cluster creates only the essential components and has no ingress or load balanced services.VALUES_URL="https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/main/starlight-for-rabbitmq/values.yaml" helm install \ --namespace datastax-pulsar \ --create-namespace \ --values $VALUES_URL \ --version 3.0.4 \ my-pulsar-cluster \ datastax-pulsar/pulsar -
Wait for the broker pod to be in a running state. You might see a few restarts as your components start up.
kubectl -n datastax-pulsar wait --for=condition=Ready pod/pulsar-broker-0 --timeout=120s
Forward service port
We will need to interact with a few of the services in the K8s cluster. Let’s map a few ports to those services.
In a new terminal, port forward Pulsar’s admin service.
kubectl port-forward -n datastax-pulsar service/pulsar-broker 8080:8080In a separate terminal window, port forward the Starlight for RabbitMQ service.
kubectl port-forward -n datastax-pulsar service/pulsar-proxy 5672:5672Produce a message with the RabbitMQ Python client
If you hadn’t noticed, we never opened the Pulsar binary port to accept new messages. Only the admin port and the RabbitMQ port are open. To further demonstrate how native Starlight for RabbitMQ is, we will use the Pika RabbitMQ Python library to produce and consume messages from Pulsar.
Save the following Python script to a safe place as test-queue.py.
The script assumes you have opened the localhost:5672 port.
#!/usr/bin/env python
import pika
connection = pika.BlockingConnection(pika.ConnectionParameters(port=5672))
channel = connection.channel()
try:
channel.queue_declare("test-queue")
print("created test-queue queue")
channel.basic_publish(exchange="", routing_key="test-queue", body="test".encode('utf-8'))
print("published message test")
_, _, res = channel.basic_get(queue="test-queue", auto_ack=True)
assert res is not None, "should have received a message"
print("received message: " + res.decode())
channel.queue_delete("test-queue")
print("deleted test-queue queue")
finally:
connection.close()Open a terminal and return to the safe place where you saved the Python script. Run the following command to execute the Python program.
python ./test-queue.pyThe output should look like the following.
created test-queue queue published message test received message: test deleted test-queue queue
Summary
Wow! That was easy.
The Luna Helm chart deployed Starlight for RabbitMQ on the Pulsar proxy and opened the correct port.
Your application will now "talk" to Pulsar as if it were a real RabbitMQ host.
