IBM Elite Support for Apache Pulsar FAQs

If you are new to IBM Elite Support for Apache Pulsar and its Apache Pulsar enhancements, these FAQs are for you.

What is IBM Elite Support for Apache Pulsar?

IBM Elite Support for Apache Pulsar is a Kubernetes-based distribution of Apache Pulsar, based on the technology that Kesque built to run its Pulsar-as-a-service.

Are IBM Elite Support for Apache Pulsar and DataStax Luna Streaming the same offering?

Yes. IBM Elite Support for Apache Pulsar is the new name for Luna Streaming after IBM acquired DataStax. In this documentation, IBM Elite Support for Apache Pulsar and Luna Streaming refer to the same product.

For technical specifications, inclusions, and exclusions of the IBM Elite Support for Apache Pulsar offering, see the IBM Elite Support for Apache Pulsar page in the IBM documentation.

What components and features are provided by IBM Elite Support for Apache Pulsar?

In addition to Apache Pulsar itself, IBM Elite Support for Apache Pulsar provides:

An installer that can stand up a dev or production cluster on bare metal or VMs without a pre-existing Kubernetes environment
A Helm chart that can deploy and manage Pulsar on your current Kubernetes infrastructure
Cassandra, Elastic, Kinesis, Apache Kafka®, and JDBC connectors
A management dashboard
A monitoring and alerting system

On which version of Apache Pulsar is IBM Elite Support for Apache Pulsar based?

IBM Elite Support for Apache Pulsar 2.10 is based on its distribution of Apache Pulsar 2.10, plus features and additional enhancements from DataStax contributors.

What does IBM Elite Support for Apache Pulsar provide that I cannot get with open-source Apache Pulsar?

IBM Elite Support for Apache Pulsar is a hardened version of Apache Pulsar that been run through additional testing to ensure it is ready for production use. It also includes additional tooling to help monitor your system, including an enhanced Admin Console and a Heartbeat service to monitor the system health.

Is IBM Elite Support for Apache Pulsar an open-source project?

Yes, IBM Elite Support for Apache Pulsar is open source. See the repos FAQ.

Which Kubernetes platforms are supported by IBM Elite Support for Apache Pulsar?

They include Minikube, K8d, Kind, Google Kubernetes Engine (GKE), Microsoft Azure Kubernetes Service (AKS), Amazon Elastic Kubernetes Service (EKS), and other commonly used platforms.

Where are the IBM Elite Support for Apache Pulsar public GitHub repos?

There are several public repos, each with a different purpose. See:

https://github.com/datastax/pulsar : This is the distro repo (a fork of apache/pulsar).
https://github.com/datastax/pulsar-admin-console : This is the repo for the Pulsar admin console, which allows for the configuration and monitoring of Pulsar.
https://github.com/datastax/pulsar-heartbeat : This is a monitoring/observability tool for Pulsar that tracks the health of the cluster and can generate alerts in Slack and OpsGenie.
https://github.com/datastax/pulsar-helm-chart : This is the Helm chart for deploying the DataStax Pulsar Distro in an existing Kubernetes cluster.
https://github.com/datastax/pulsar-sink : This is the DataStax Apache Pulsar Connector (pulsar-sink) repo.
https://github.com/datastax/burnell : This is a utility for Pulsar that provides various functions, such as key initialization for authentication, and JWT token creation API.

Is there a prerequisite version of Java needed for the IBM Elite Support for Apache Pulsar installation?

The IBM Elite Support for Apache Pulsar distribution is designed for Java 11. However, because the product releases Docker images, you do not need to install Java (8 or 11) in advance. Java 11 is bundled in the Docker image.

What are the install options for IBM Elite Support for Apache Pulsar?

DataStax Pulsar Helm chart: Use the Helm chart provided at https://github.com/datastax/pulsar-helm-chart to install IBM Elite Support for Apache Pulsar in an existing Kubernetes cluster on your laptop or hosted by a cloud provider.
Tarball: Use the tarball provided at https://github.com/datastax/pulsar/releases to install IBM Elite Support for Apache Pulsar on a server or VM.
Ansible: Use the DataStax Ansible scripts provided at https://github.com/datastax/pulsar-ansible to install IBM Elite Support for Apache Pulsar on a server or VM with our provided playbooks.

How do I install IBM Elite Support for Apache Pulsar in my Kubernetes cluster?

Follow the full instructions in Quick Start for Helm Chart installs.

How do I install IBM Elite Support for Apache Pulsar on my server or VM?

Follow the full instructions in Quick Start for Bare Metal/VM installs.

What task can I perform in the IBM Elite Support for Apache Pulsar Admin Console?

From the Admin Console, you can:

Add and run Pulsar clients
Establish credentials for secure connections
Define topics that can be published for streaming apps
Set up Pulsar sinks that publish topics and make them available to subscribers, such as for a Cassandra database table
Control namespaces used by Pulsar
Use the Admin API

What is Pulsar Heartbeat?

Pulsar Heartbeat monitors the availability, tracks the performance, and reports failures of the Pulsar cluster. It produces synthetic workloads to measure end-to-end message pubsub latency. Pulsar Heartbeat is a cloud-native application that can be installed by Helm within the Pulsar Kubernetes cluster.

What are the features provided by DataStax Apache Pulsar Connector (`pulsar-sink`) that are not supported in `kafka-sink`?

The Pulsar IO framework provides many features that are not possible in Kafka, and has different compression formats and auth/security features. The features are handled by Pulsar. For more, see IBM Elite Support for Apache Pulsar IO Connectors.

The DataStax Apache Pulsar Connector allows single-record acknowledgement and negative acknowledgements.

What features are missing in DataStax Apache Pulsar Connector (`pulsar-sink`) compared with `kafka-sink`?

No support for tinyint (int8bit) and smallint (int16bit).
The key is always a String, but you can write JSON inside it; the support is implemented in pulsar-sink, but not in Pulsar IO.
The “value” of a “message property” is always a String; for example, you cannot map the message property to ttl or to timestamp.
Field names inside structures must be valid for Avro, even in case of JSON structures. For example, field names like Int.field (with dot) or int field (with space) are not valid.

How is DataStax Apache Pulsar Connector distributed?

There are two packages:

The pulsar-sink functionality of DataStax Apache Pulsar Connector is included with IBM Elite Support for Apache Pulsar. It’s built in!
You can optionally download the DataStax Apache Pulsar Connector .nar file from the pulsar-sink repository, and then use it as its own product with your open-source Apache Pulsar install.

If you’re using open-source software (OSS) Apache Pulsar, you can use DataStax Apache Pulsar Connector with the OSS to take advantage of this pulsar-sink for Cassandra. See the DataStax Apache Pulsar Connector documentation.

What is the DataStax Change Data Capture (CDC) for Cassandra connector?

This source connector streams data changes from Cassandra tables to Pulsar topics. For more information, see the DataStax CDC for Cassandra connector documentation.

What client APIs does IBM Elite Support for Apache Pulsar provide?

The same as for Apache Pulsar. See https://pulsar.apache.org/docs/en/client-libraries/.