Luna Streaming Installation on Azure Kubernetes Service

This document covers installation of Luna Streaming on the Azure Kubernetes Service (AKS).

For an example set of production cluster values, see the DataStax production-ready Helm chart.

DataStax recommends these hardware resources for running Luna Streaming in a Kubernetes environment:

Helm version 3
A Kubernetes cluster
Two node pools
- One function-worker node pool for deploying sink and source connectors, and the other node pool for everything else
Must use SSD disks
Depending on the cloud provider, the latest 'Storage Driver' should be used, along with the fastest disk type (for example, GP3 in AWS)
5 Zookeeper replicas
3 Bookies
3 Brokers
3 Proxies

For the local machine running the Helm chart, you will need:

Helm version 3
An Azure account with privileges to install and configure Kubernetes resources.

Helm chart `values.yaml`

The values.yaml will deploy Pulsar with:

1 ZooKeeper cluster with replicaCount:3
1 Bookkeeper cluster with replicaCount:3
1 Broker cluster with replicaCount: 3
- Ledgers: defaultEnsembleSize: 2, defaultAckQuorum: 2, defaultWriteQuorum: 2
1 Function cluster with a replicaCount: 2

`default_storage`

The default_storage parameter in values.yaml controls the default storage class for all persistent volumes created by the Helm chart.

default_storage:
  existingStorageClassName: default

For a component like BookKeeper, which requires stateful storage, we need to override the default_storage class when the BookKeeper Persistent Volume Claims (PVCs) are created.

There are two ways to override default_storage:

Leave existingStorageClassName blank and specify the storage class parameters below.

  volumes:
    journal:
      name: journal
      size: 20Gi
      existingStorageClassName:
        storageClass:
         provisioner: kubernetes.io/azure-disk
         type: pd-ssd
         fsType: ext4
         extraParams:
            replication-type: none

Create a custom storage configuration as a yaml file (like the DataStax example) and tell the Helm chart to use that storage configuration when it creates the BookKeeper PVCs.
```
  volumes:
    journal:
      name: journal
      size: 20Gi
      existingStorageClassName: bookkeeper-storageclass.yaml
```

AKS-specific storage class

Here is an example storage-values.yaml as used above for the BookKeeper node. Paste it into a yaml file to provision a persistent SSD with the ext4 file system.

storage_class:
   provisioner: kubernetes.io/azure-disk
   fsType: ext4
   type: managed-premium
   extraParams:
     storageaccounttype: Premium_LRS
     kind: Managed
     cachingmode: ReadOnly

Install Luna Streaming to AKS

Once you have added your storage settings to the storage-values.yaml file, install the Helm chart.

Create the namespace; in this example, we use pulsar:
```
kubectl create namespace pulsar
```

Run this Helm command using the storage-values.yaml provisioning:

helm install pulsar datastax-pulsar/pulsar --namespace pulsar --values storage_values.yaml --create-namespace`

To avoid having to specify the pulsar namespace on each subsequent command, set the namespace context:
```
kubectl config set-context $(kubectl config current-context) --namespace=pulsar
```
Execute kubectl get pods -A to view your running pods.
Once your pods are up and running, continue to Accessing Pulsar Cluster in Cloud to access and manage your cluster.

What’s next?

Once your pods are up and running, continue to Accessing Pulsar Cluster in Cloud to access and manage your cluster.

To learn more about using the Pulsar Admin Console, see Admin Console Tutorial.
To learn more about installing Pulsar on a server or virtual machine, see Quickstart for Server/VM Install.

Was this helpful?

Edit this page