• Glossary
  • Support
  • Downloads
  • DataStax Home
Get Live Help
Expand All
Collapse All

DataStax Pulsar Connector

    • Getting Started
      • About the DataStax Apache Pulsar™ Connector
        • System requirements
      • Pulsar Connector release notes
      • Installing DataStax Apache Pulsar™ Connector
      • Pulsar Connector single instance quick start
    • Guides and examples
      • Configuration
        • Configuring parallelism
        • Specify writetime timestamp column
        • Setting row-level TTL values from Pulsar fields
        • Pass Pulsar Connector settings directly to the DataStax Java driver
        • Mapping pulsar topics to database tables
          • Determining topic data structure
          • Mapping basic messages to table columns
          • Mapping a message that contain JSON fields
            • Mapping a message that contains both basic and JSON fields
            • Mapping JSON messages
          • Mapping Avro messages
          • Extract Pulsar record header values
          • Mapping messages to table that has a User Defined Type
          • Mapping a topic to multiple tables
          • Multiple topics to multiple tables
          • Provide CQL queries in mappings
          • The now() function in mappings
      • Operations
        • About operating and maintaining the DataStax Connector
        • Scaling the DataStax Apache Pulsar™ Connector
        • Changing the topic or table schema
        • Restarting the DataStax Apache Pulsar™ Connector
        • Displaying the DataStax Apache Pulsar™ Connector configuration
        • Updating the DataStax Apache Pulsar™ Connector configuration
        • Deleting the DataStax Apache Pulsar™ Connector
        • Getting the DataStax Connector status
      • Security
        • Using internal or LDAP authentication
      • DataStax Apache Pulsar™ Connector metrics
      • Troubleshooting
        • Record fails to write
        • Writing fails because of mutation size
        • Data parsing fails
        • Loading balancing datacenter is not specified
    • Reference
      • DataStax Apache Pulsar™ Connector details
      • DataStax connection
      • Pulsar topic-to-table settings
      • Converting date and times for a topic
      • Using the DataStax Apache Pulsar™ Connector with DataStax Enterprise authentication
        • Internal or LDAP authentication
      • SSL encrypted connection
      • Configure error handling
  • DataStax Pulsar Connector
  • Guides and examples
  • Configuration
  • Setting row-level TTL values from Pulsar fields

Setting row-level TTL values from Pulsar fields

In the Apache Pulsar™ topic mapping, you can optionally specify which column should be used as the ttl (time-to-live) of the record being inserted into the database table. The mapping setting provides a special property, __ttl. The format is:

topic:
  my_topic:
    my_ks:
      my_table:
        mapping: 'col1=key.f1, col2=value.f1, __ttl=value.f2'

When you specify a ttl value to be used while inserting the record, by mapping __ttl or .ttl static configuration, you can further specify a timeUnit. The default is SECONDS. If you want to specify a ttl value in a unit such as HOURS, set timeUnit to HOURS. DataStax Apache Pulsar Connector automatically transforms the value to SECONDS, which is the DSE ttl unit. The transformation also applies for static .ttl field configuration:

topic:
  my_topic:
    my_ks:
      my_table:
        ttlTimeUnit: SECONDS

If the ttl is provided via mapping, such as mapping: `__ttl=value.f2, and by the ttl static configuration (ttl: 86400) the ttl value from the mapping takes precedence and is used. If you set ttl in both resources, DataStax Apache Pulsar™ Connector writes a warning message in the log file.

When you configure ttl, all rows for that topic table will have this same TTL value. DataStax Apache Pulsar™ Connector appends AND TTL <configured-ttl-value> to the INSERT statement for those rows.

When doing dynamic configuration, if the ttl field value is negative, an error is thrown during the insert of the record into a database table. However, because the Pulsar connector is a streaming system, DataStax Apache Pulsar Connector resorts to inserting the record without the ttl value.
Specify writetime timestamp column Pass Pulsar Connector settings directly to the DataStax Java driver

General Inquiries: +1 (650) 389-6000 info@datastax.com

© DataStax | Privacy policy | Terms of use

DataStax, Titan, and TitanDB are registered trademarks of DataStax, Inc. and its subsidiaries in the United States and/or other countries.

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Kubernetes is the registered trademark of the Linux Foundation.

landing_page landingpage