Pulsar topic-to-table settings

Capture Apache Pulsar™ topics in the supported database by specifying a target keyspace, table, and then map the topic fields to table columns in the name of the parameter.

Parameters

Use the following syntax for standalone properties file:

topics: topic_list
topic:
  topic_name:
    keyspace_name:
      table_name:
        mapping: mapping_specification
        consistencyLevel: WRITE_CONSISTENCY_LEVEL
        ttlTimeUnit: seconds
        ttl: -1
        nullToUnset: true
        deletesEnabled: true
topics

A comma separated list of all topics to which the DataStax Connector subscribes.

where:

  • topic_name - Pulsar topic name.

  • keyspace_name - database keyspace where the table is located.

  • table_name - database table where data is written.

mapping

Required, field-to-column mapping. See Mapping pulsar topics to database tables.

consistencyLevel

Query consistency level. DSE settings are:

nullToUnset

Whether to treat nulls in Pulsar as UNSET in DSE. DataStax recommends using the default to avoid creating unnecessary tombstones.

Default: true

ttl

Time-to-live. Set to the number of seconds before the data is automatically deleted from the DSE table. When you configure topic.<topic-name>.<keyspace-name>.<table-name>.ttl, all rows for that topic table will have this same TTL value. DataStax Pulsar Connector appends AND TTL <configured-ttl-value> to the INSERT statement for those rows.

Default: -1 (disabled)

deletesEnabled

When enabled, treat records that after mapping would result in only non-null values for primary key columns as deletes, rather than inserting/updating nulls for all regular columns.

Only triggers if the setting is enabled and the mapping specification contains all DSE table columns.

Default: true