System requirements

The system requirements for DataStax Apache Pulsar™ Connector depends on the workload and network capacity. The factors include characteristics of the Pulsar topic and the cluster data models and volume. DataStax recommends testing with realistic data flows before committing to an instance type for the connector.

DataStax Apache Pulsar™ Connector is bound by the amount of CPU available on the host. The connector holds all the records pulled from Pulsar topics in memory, along with the cluster metadata and prepared statements.

Memory pressure is influenced by:

  • Record size of Pulsar topics

  • Number of records pulled at the same time, where the maximum is set by the workers batchSize parameter.

  • Number of simultaneous tasks run by the connector

DataStax Apache Pulsar™ Connector needs adequate network capacity for the payload. This includes the connections from Pulsar Servers to the target platform. Scale the connector horizontally by adding additional workers to increase overall throughput.

The DataStax Apache Pulsar™ Connector framework automatically rebalances the load when workers are added by reallocating the tasks among the workers.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000,