Configure the Starlight for Kafka extension

This page describes configuration options for Starlight for Kafka.

Listeners

This section lists configurations for topic listeners.

Name Description

kafkaListeners

Required comma-separated list of URIs that Starlight for Kafka listens on and the listener names. For example: PLAINTEXT://localhost:9092,SSL://localhost:9093

Each URI’s scheme represents a listener name if kafkaProtocolMap is configured. Otherwise, the scheme must be a valid protocol in PLAINTEXT, SSL, SASL_PLAINTEXT, or SASL_SSL.

If the hostname isn’t set, it is bound to the default interface.

kafkaAdvertisedListeners

Optional listeners to publish to ZooKeeper for clients to use. The format is the same as kafkaListeners.

kafkaProtocolMap

Optional comma-separated map of listener name and protocol. For example: PRIVATE:PLAINTEXT,PRIVATE_SSL:SSL,PUBLIC:PLAINTEXT,PUBLIC_SSL:SSL

listeners

Deprecated. Use kafkaListeners instead.

Multiple listeners

To support multiple listeners, specify different listener names in kafkaListeners and kafkaAdvertisedListeners, then map the listener name to the proper protocol in kafkaProtocolMap.

For example, to listen on ports 9092 and 19092 with the PLAINTEXT protocol, the associated names are kafka_internal and kafka_external.

Add the following configurations:

kafkaListeners=kafka_internal://0.0.0.0:9092,kafka_external://0.0.0.0:19092
kafkaProtocolMap=kafka_internal:PLAINTEXT,kafka_external:PLAINTEXT
kafkaAdvertisedListeners=kafka_internal://localhost:9092,kafka_external://localhost:19092

In the above example,

  • kafkaListener is split into multiple tokens by a comma (,), the token is in a format of <listener-name>://<host>:<port>.

  • kafkaProtocolMap is split into multiple tokens by a comma (,), the token is in a format of <listener-name>:<protocol>.

  • kafkaAdvertisedListeners is split into multiple tokens by a comma(,), the token is in a format of <listener-name>:<scheme>://<host>:<port>.

Logger

Starlight for Kafka shares the same configuration files with the Pulsar broker, e.g. conf/broker.conf or conf/standalone.conf. The log configuration is modified in conf/log4j2.yaml file as below:

Logger:
  - name: io.streamnative.pulsar.handlers.kop
    level: warn
    additivity: false
    AppenderRef:
      - ref: Console

Namespace

Pulsar is a multi-tenant system that requires users to specify the tenant and namespace. While most Kafka users just specify the short topic name, Starlight for Kafka provides the following configurations to specify the default namespace.

Name Description Default

kafkaTenant

The default tenant of Kafka topics

public

kafkaNamespace

The default namespace of Kafka topics

default

kafkaMetadataTenant

The tenant used for storing Kafka metadata topics

public

kafkaEnableMultiTenantMetadata

Use the SASL username as kafkaMetadataTenant

true

kafkaMetadataNamespace

The namespace used for storing Kafka metadata topics

kafka

kopSchemaRegistryNamespace

The namespace used for storing Kafka metadata topics

kafka_schemaregistry

kafkaManageSystemNamespaces

Control whether the system should automatically manage the namespaces.

true

kopAllowedNamespaces

The allowed namespace to list topics with a comma separator. e.g. public/default,public/kafka.

If empty or not set, then the allowed namespaces are <kafkaTenant>/<kafkaNamespace>.

Enable kafkaEnableMultiTenantMetadata to use separate tenants when handling system metadata. This enhancement results in fully isolated tenants in the Pulsar cluster, which isn’t available in pure Kafka where the system metadata is shared among all users.

Performance

This section lists configurations that can affect performance.

Name Description Default

maxReadEntriesNum

The maximum number of entries that are read from the cursor once per time. Increasing this value can make FETCH requests read more bytes each time.

Starlight for Kafka doesn’t check the maximum byte limit. Therefore, if this value is too large, then the response size can exceed the network limit.

5

Network

This section lists configurations for network communication.

These limits are based on each connection.

Name Description Default

maxQueuedRequests

Limit the queue size for request, like queued.max.requests in Kafka server.

500

requestTimeoutMs

Limit the timeout in milliseconds for request, like request.timeout.ms in Kafka client. If a request wasn’t processed within the timeout limit, then Starlight for Kafka returns an error response to the client.

30000

connectionMaxIdleMs

The idle connection timeout in milliseconds. If the idle connection timeout (such as connections.max.idle.ms used in the Kafka server) is reached, then the server handler closes this idle connection.

If set to -1, the idle connection timeout is disabled.

600000

failedAuthenticationDelayMs

Connection close delay on failed authentication: This is the time (in milliseconds) by which connection close is delayed on authentication failure, like connection.failed.authentication.delay.ms in Kafka server.

300

brokerLookupTimeoutMs

The timeout for broker lookups (in milliseconds).

30000

Prometheus

This section lists configurations for Prometheus metrics.

Name Description Default

kopPrometheusStatsLatencyRolloverSeconds

Starlight for Kafka metrics exposed to prometheus rollover latency in seconds.

60

Group Coordinator

This section lists configurations about the group coordinator and the __consumer_offsets topic that is used to store committed offsets.

Name Description Default

groupMinSessionTimeoutMs

The minimum allowed session timeout for registered consumers. Shorter timeouts result in quicker failure detection while require more frequent consumer heart beating, which can overwhelm broker resources.

6000

groupMaxSessionTimeoutMs

The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages between heartbeats while require longer time to detect failures.

300000

groupInitialRebalanceDelayMs

The time the group coordinator waits for more consumers to join a new group before performing the first rebalance. A longer delay potentially reduces rebalances, but increases the time until processing begins.

3000

offsetsTopicCompressionCodec

Compression codec for the offsets topic.

NONE

offsetMetadataMaxSize

The maximum size in bytes for a metadata entry associated with an offset commit.

4096

offsetsRetentionMinutes

Offsets older than this retention period are discarded.

4320

offsetsMessageTTL

The offsets message TTL in seconds.

259200

offsetsRetentionCheckIntervalMs

The frequency at which to check for stale offsets.

600000

offsetsTopicNumPartitions

The number of partitions for the offsets topic.

50

Transactions

This section lists configurations for transactions.

Name Description Default

enableTransactionCoordinator

Whether to enable transaction coordinator.

false

brokerId

The broker ID that is used to create the producer ID.

1

txnLogTopicNumPartitions

the number of partitions for the transaction log topic.

50

txnAbortTimedOutTransactionCleanupIntervalMs

The interval in milliseconds at which to rollback transactions that have timed out.

10000

enableTransactionalIdExpiration

Whether to enable transactional ID expiration.

true

transactionalIdExpirationMs

The time (in ms) that the transaction coordinator waits without receiving any transaction status updates for the current transaction before expiring its transactional ID.

604800

transactionsRemoveExpiredTransactionalIdCleanupIntervalMs

The interval (in ms) at which to remove expired transactions.

3600

SSL encryption

This section lists configurations for SSL encryption.

Name Description Default

kopSslProtocol

Kafka SSL configuration map with: SSL_PROTOCOL_CONFIG = ssl.protocol

TLS

kopSslProvider

Kafka SSL configuration map with: SSL_PROVIDER_CONFIG = ssl.provider

kopSslCipherSuites

Kafka SSL configuration map with: SSL_CIPHER_SUITES_CONFIG = ssl.cipher.suites

kopSslEnabledProtocols

Kafka SSL configuration map with: SSL_ENABLED_PROTOCOLS_CONFIG = ssl.enabled.protocols

TLSv1.2, TLSv1.1, TLSv1

kopSslKeystoreType

Kafka SSL configuration map with: SSL_KEYSTORE_TYPE_CONFIG = ssl.keystore.type

JKS

kopSslKeystoreLocation

Kafka SSL configuration map with: SSL_KEYSTORE_LOCATION_CONFIG = ssl.keystore.location

kopSslKeystorePassword

Kafka SSL configuration map with: SSL_TRUSTSTORE_PASSWORD_CONFIG = ssl.truststore.password

N/A

kopSslTruststoreType

Kafka SSL configuration map with: SSL_KEYSTORE_TYPE_CONFIG = ssl.keystore.type

JKS

kopSslTruststoreLocation

Kafka SSL configuration map with: SSL_TRUSTSTORE_LOCATION_CONFIG = ssl.truststore.location

kopSslTruststorePassword

Kafka SSL configuration map with: SSL_TRUSTSTORE_PASSWORD_CONFIG = ssl.truststore.password

kopSslKeymanagerAlgorithm

Kafka SSL configuration map with: SSL_KEYMANAGER_ALGORITHM_CONFIG = ssl.keymanager.algorithm

SunX509

kopSslTrustmanagerAlgorithm

Kafka SSL configuration map with: SSL_TRUSTMANAGER_ALGORITHM_CONFIG = ssl.trustmanager.algorithm

SunX509

kopSslSecureRandomImplementation

Kafka SSL configuration map with: SSL_SECURE_RANDOM_IMPLEMENTATION_CONFIG = ssl.secure.random.implementation

See also

Was this helpful?

Edit this page

Give Feedback

How can we improve the documentation?

© Copyright IBM Corporation 2026 | Privacy policy | Terms of use |  Manage Privacy Choices

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: Contact IBM