• Glossary
  • Support
  • Downloads
  • DataStax Home
Get Live Help
Expand All
Collapse All

DataStax Pulsar Connector

    • Getting Started
      • About the DataStax Apache Pulsar™ Connector
        • System requirements
      • Pulsar Connector release notes
      • Installing DataStax Apache Pulsar™ Connector
      • Pulsar Connector single instance quick start
    • Guides and examples
      • Configuration
        • Configuring parallelism
        • Specify writetime timestamp column
        • Setting row-level TTL values from Pulsar fields
        • Pass Pulsar Connector settings directly to the DataStax Java driver
        • Mapping pulsar topics to database tables
          • Determining topic data structure
          • Mapping basic messages to table columns
          • Mapping a message that contain JSON fields
            • Mapping a message that contains both basic and JSON fields
            • Mapping JSON messages
          • Mapping Avro messages
          • Extract Pulsar record header values
          • Mapping messages to table that has a User Defined Type
          • Mapping a topic to multiple tables
          • Multiple topics to multiple tables
          • Provide CQL queries in mappings
          • The now() function in mappings
      • Operations
        • About operating and maintaining the DataStax Connector
        • Scaling the DataStax Apache Pulsar™ Connector
        • Changing the topic or table schema
        • Restarting the DataStax Apache Pulsar™ Connector
        • Displaying the DataStax Apache Pulsar™ Connector configuration
        • Updating the DataStax Apache Pulsar™ Connector configuration
        • Deleting the DataStax Apache Pulsar™ Connector
        • Getting the DataStax Connector status
      • Security
        • Using internal or LDAP authentication
      • DataStax Apache Pulsar™ Connector metrics
      • Troubleshooting
        • Record fails to write
        • Writing fails because of mutation size
        • Data parsing fails
        • Loading balancing datacenter is not specified
    • Reference
      • DataStax Apache Pulsar™ Connector details
      • DataStax connection
      • Pulsar topic-to-table settings
      • Converting date and times for a topic
      • Using the DataStax Apache Pulsar™ Connector with DataStax Enterprise authentication
        • Internal or LDAP authentication
      • SSL encrypted connection
      • Configure error handling
  • DataStax Pulsar Connector
  • Guides and examples
  • Configuration
  • Pass Pulsar Connector settings directly to the DataStax Java driver

Pass Pulsar Connector settings directly to the DataStax Java driver

In your DataStax Apache Pulsar™ Connector configuration file, you can directly pass settings to the DataStax Java driver by using the datastax-java-driver prefix. For example:

datastax-java-driver.basic.request.consistency=ALL

Mapping Pulsar Connector settings to Java driver properties

The following table identifies functionally equivalent DataStax Apache Pulsar Connector and DataStax Java driver settings.

If you define both in your configuration, the Pulsar Connector setting take precedence over the datastax-java-driver.property-name. If you do not provide either in your configuration, Pulsar Connector defaults are in effect.

For information about the Java properties, refer to the DataStax Java driver documentation. For information about the Pulsar Connector settings, refer to DataStax connection.

DataStax Apache Pulsar Connector setting Using datastax-java-driver prefix

contactPoints

datastax-java-driver.basic.contact-points

loadBalancing.localDc

datastax-java-driver.basic.load-balancing-policy.local-datacenter

cloud.secureConnectBundle

datastax-java-driver.basic.cloud.secure-connect-bundle

queryExecutionTimeout

datastax-java-driver.basic.request.timeout

connectionPoolLocalSize

datastax-java-driver.advanced.connection.pool.local.size

compression

datastax-java-driver.advanced.protocol.compression

metricsHighestLatency

datastax-java-driver.advanced.metrics.session.cql-requests.highest-latency

There is a difference between the Pulsar Connector’s contactPoints setting and the Java driver’s datastax-java-driver.basic.contact-points. For contactPoints, the value of the port is appended to every host provided by this setting. For datastax-java-driver.basic.contact-points, you must provide the fully qualified contact points (host:port).

By passing in the Java driver’s setting, this option gives you more configuration flexibility because you can specify a different port for each host. Example:

datastax-java-driver.basic.contact-points = 127.0.0.1:9042, 127.0.0.2:90

Conversion of Java driver properties of type List to TypeSafe Config

The following properties that are of type List, which you could pass into the driver from your Pulsar Connector configuration via the datastax-java-driver prefix, are converted by the DataStax Java driver to the TypeSafe Config format.

  • datastax-java-driver.advanced.ssl-engine-factory.cipher-suites

  • datastax-java-driver.advanced.metrics.node.enabled

  • datastax-java-driver.advanced.metadata.schema.refreshed-keyspaces

  • datastax-java-driver.advanced.metrics.session.enabled

  • datastax-java-driver.basic.contact-points

The conversion by the Java driver will split the comma-separated values provided in those setting to create indexed properties. For example, if you passed in datastax-java-driver.advanced.metrics.session.enabled=a,b, the entry is converted to:

datastax-java-driver.advanced.metrics.session.enabled.0=a
datastax-java-driver.advanced.metrics.session.enabled.1=b

For more information, refer to the Java driver reference configuration topic.

Setting row-level TTL values from Pulsar fields Mapping pulsar topics to database tables

General Inquiries: +1 (650) 389-6000 info@datastax.com

© DataStax | Privacy policy | Terms of use

DataStax, Titan, and TitanDB are registered trademarks of DataStax, Inc. and its subsidiaries in the United States and/or other countries.

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Kubernetes is the registered trademark of the Linux Foundation.

landing_page landingpage