• Glossary
  • Support
  • Downloads
  • DataStax Home
Get Live Help
Expand All
Collapse All

DataStax Streaming Home

Luna Streaming Documentation

    • Install & Upgrade
      • Quick Start for Helm Chart installs
      • Quick Start for Bare Metal/VM installs
      • Supported Software
      • Production Cluster Sizing
        • Cluster Sizing Reference
    • Components & Extensions
      • Pulsar Admin Console
      • Install Pulsar Admin Console on Server/VM
      • Pulsar Heartbeat
      • Heartbeat on VM/Server
      • Pulsar Beam
      • Pulsar SQL
      • Starlight for Kafka
      • Starlight for RabbitMQ
    • Operations
      • Luna Streaming Authentication
      • Luna Streaming Functions
      • Luna Streaming IO connectors
      • Scaling your Luna Streaming cluster
      • Troubleshooting
👀 Looking for examples and
use cases?
Visit our learning site
  • Luna Streaming Documentation
  • Install & Upgrade
  • Production Cluster Sizing
  • Cluster Sizing Reference
2.10-3.x 2.10-2.x 2.10-1.x
Edit this Page

Installation topologies

This page describes recommended starting points for Pulsar deployments.

  • SANDBOX (or Pulsar Standalone) is an all-in-one single-node deployment that is useful for taking Pulsar for a test drive.

  • DEVELOPMENT is a 3-node deployment that is not highly available, but able to maintain parity with the TESTING environment.

  • SINGLE REGION TESTING ENVIRONMENT is a highly-available 3/3/3 deployment.

  • HIGH-AVAILABILITY PRODUCTION ENVIRONMENT is a highly-available deployment replicated across 3 zones in 1 region.

The number of nodes in a highly available cluster depends on the number of zones in the region. We recommend aligning the replication factor with the AZ count, so in this example, 3 zones and a replication factor of 3.

Environment Server Component Node Count Note

SANDBOX (STANDALONE)

(combined) Zookeeper, Broker, and Bookie

1

DEVELOPMENT (1 region, 1 zone)

Zookeeper

1

Broker

1

Bookie

1

SINGLE REGION TESTING ENVIRONMENT (1 region, 1 zone)

Zookeeper

3

Broker

3

Bookie

3

Pulsar proxy

3

(Dedicated) Function Worker

3

HIGH-AVAILABILITY PRODUCTION ENVIRONMENT (1 region, 3 zones)

Zookeeper

5

2/2/1 nodes*

Broker

3

1 nodes per AZ*

(Dedicated) Function Worker

3

1 node per AZ*
The number of function workers depends on the cluster’s functions workload.

Bookie

6

2 nodes per AZ*

Pulsar proxy

3

1 node per AZ*

Autorecovery

3

1 per AZ*

*Assuming a replication factor of 3.

Hardware sizing

The following table lists the minimum hardware requirements for a Pulsar cluster.

Environment Server Component Hardware Spec Note

DEVELOPMENT

Zookeeper, Broker, and Bookkeeper

  • CPU: 8 vCPU

  • Memory: 16 GB

  • Data Disk: 64 GB SSD

Data disks are shared by:
* Zookeeper data
* Bookkeeper journal
* Bookkeeper ledger

TEST

Zookeeper

  • CPU: 2 vCPU

  • Memory: 4 GB

  • Data Disk: 32 GB SSD

Broker and Function Worker

  • CPU: 8 vCPU

  • Memory: 16 GB

No data disk

Bookie

  • CPU: 4 vCPU

  • Memory: 16 GB

  • Data Disk

    • Journal: 32 GB SSD

    • Ledger: 256 GB SSD

Pulsar Proxy, Function Worker

  • CPU: 4 vCPU

  • Memory: 8 GB

PRODUCTION

Zookeeper

  • CPU: 4 vCPU

  • Memory: 4 GB

  • Data Disk: 64 GB SSD

Broker

  • CPU: 8 vCPU

  • Memory: 16 GB

  • No data disk

Bookie

  • CPU: 8 vCPU

  • Memory: 32 GB

  • Data Disk

    • Journal: 256 GB SSD

    • Ledger: 1024 GB SSD

Ledger disk capacity can be beyond 1TB.

Pulsar Proxy, Autorecovery

  • CPU: 4 vCPU

  • Memory: 16 GB

Dedicated Function Worker

  • CPU: 4 vCPU

  • Memory: 8 GB

What’s next

For more on cluster sizing, see Production Cluster Sizing.

Production Cluster Sizing Components & Extensions

General Inquiries: +1 (650) 389-6000 info@datastax.com

© DataStax | Privacy policy | Terms of use

DataStax, Titan, and TitanDB are registered trademarks of DataStax, Inc. and its subsidiaries in the United States and/or other countries.

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Kubernetes is the registered trademark of the Linux Foundation.

landing_page landingpage