Starlight for Kafka schema registry

Starlight for Kafka implements a schema registry that is compatible with the Confluent Schema Registry® and the Apicurio Schema Registry, the most commonly used schema registries in the Kafka ecosystem.

The Kafka Schema Registry stores schemas in the $<tenant>/__kafka/__schema-registry topic. It is automatically created as a non-partitioned topic with infinite retention.

To avoid losing data, it is critical that you disable retention on the __kafka namespace. Or, if you have topic-level policies enabled in your Pulsar environment, set it for that topic only.

The Starlight for Kafka adapter creates automatically the $<tenant>/__kafka__schemaregistry namespace and sets infinite retention for you.

You can use kopSchemaRegistryNamespace configuration parameter to set the name of the namespace to be used.

Enable the schema registry

To enable the schema registry, add this configuration in conf/broker.conf:
```
kopSchemaRegistryEnable=true
kopSchemaRegistryProxyPort=8001
```
If you’re using the Kafka proxy, add this configuration in conf/proxy.conf:
```
kopSchemaRegistryEnable=true
kopSchemaRegistryProxyPort=8081
```
Verify that the Schema Registry is working properly by connecting to http://pulsar-broker:8001/config or http://pulsar-proxy:8081/config (if you’re using the proxy). You should see something like this:
```
{
  "compatibilityLevel" : "NONE"
}
```
The Proxy works only if the broker is working properly because it only forwards the requests to a broker. If the broker fails or it isn’t accessible, then the Proxy answers with an error message.

Enable TLS on the schema registry

To enable TLS on the schema registry, you must use the Kafka proxy.

To set the SchemaRegistry endpoint on the broker to use TLS, add this configuration in conf/broker.conf:
```
kopSchemaRegistryEnableTls=true
kopSchemaRegistryProxyEnableTls=true
```
To set the SchemaRegistry endpoint on the proxy to use TLS, add this configuration in conf/proxy.conf:
```
kopSchemaRegistryEnableTls=true
kopSchemaRegistryProxyEnableTls=true
```
TLS configuration is then picked up automatically by the Pulsar proxy configuration.

Authentication for the schema registry

If authentication is enabled, you can only use the schema registry with HTTP Basic Authentication:

Username: the name of the tenant to connect to
Password: token:xxxxxxxx (the JWT token used to connect to Pulsar/Kafka)

If you are using the Confluent Java client or the Kafka Tools, set these properties:

basic.auth.credentials.source=USER_INFO
basic.auth.user.info=TENANTNAME:token:xxxxxxxx
schema.registry.url=http://pulsar-proxy:8081

Starlight for Kafka schema registry

Enable the schema registry

Enable TLS on the schema registry

Authentication for the schema registry

See also

Was this helpful?

Give Feedback