Starlight for Kafka proxy extension

Starlight for Kafka allows you to deploy a proxy extension on the Pulsar proxy component. This allows the Kafka client to access your Pulsar cluster the same way as Pulsar clients do.
This is particularly useful in Kubernetes environments where you already have the Pulsar proxy.


  1. Create a proxyextensions folder in the root of your Pulsar directory.

  2. Copy the pulsar-kafka-proxy- file to the “proxyextensions” directory.

  3. Add these lines to proxy.conf:

    # Local listener
    # Advertised listener to the clients
    # TLS settings

kafkaAdvertisedListeners must contain the public address that clients will use to connect to the proxy. In the example above we are using pulsar-proxy:9092, but this address is available only inside the Kubernetes cluster. If you are exposing your service outside of the Kubernetes cluster, you must use the public name.

TLS Configuration for the Proxy

TLS is configured using the same TLS configuration as the Pulsar proxy. To expose TLS endpoints, change the following settings in conf/proxy.conf.

kafkaListeners=PLAINTEXT://, SSL://
kafkaAdvertisedListeners=PLAINTEXT://pulsar-proxy:9092, SSL://pulsar-proxy:9093

The proxy always uses PLAINTEXT connection while connecting to the internal brokers, so if you are configuring TLS on the proxy you must also configure a PLAINTEXT listener on the broker.

Authentication and authorization for the Proxy

In order to configure authentication and authorization for the proxy you must enable authentication and authorization on the Pulsar proxy. The Kafka proxy will use the same configuration as the Pulsar proxy.

Add these lines to the Pulsar proxy:


The first line tells the proxy to accept username/password authentication. In the second line, ‘admin’ is the name of a “role” that is allowed to perform administrative operations on the cluster. This role is needed to perform authorization tasks on the proxy, like validating the user that is logging in.

If you enable authentication and authorization on the proxy then you must also enable them on the broker, and the Protocol Handler must be configured to listen on PLAINTEXT_SASL.

Discovering Pulsar Brokers from the Proxy

The proxy uses the broker discovery service to discover the brokers. The Pulsar broker does not advertise the address of the Kafka listeners, so the mapping between a broker and the actual TCP port that is listening for Kafka connections is done per convention.
If a Pulsar broker exposes the Pulsar endpoint at port 6650, the proxy assumes that it is exposing the Kafka endpoint at port 9092. The same applies for TLS communications, where port 6651 is mapped to 9093. You can override this mapping by using the kafkaProxyBrokerPortToKopMapping configuration entry:


This means that a broker on port 6650 for Pulsar protocol will be mapped to Kafka port 19092 and port 6651 is mapped to 19093. This is usually not needed for standard deployments that use default ports.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000,