Pulsar topic-to-table settings

Capture Apache Pulsar™ topics in the supported database by specifying a target keyspace, table, and then map the topic fields to table columns in the name of the parameter.

Parameters

Use the following syntax for standalone properties file:

topics: topic_list
topic:
  topic_name:
    keyspace_name:
      table_name:
        mapping: mapping_specification
        consistencyLevel: WRITE_CONSISTENCY_LEVEL
        ttlTimeUnit: seconds
        ttl: -1
        nullToUnset: true
        deletesEnabled: true

topics

A comma separated list of all topics to which the DataStax Connector subscribes.

topic_name - Pulsar topic name.
keyspace_name - database keyspace where the table is located.
table_name - database table where data is written.

mapping

Required, field-to-column mapping.

For more, see Mapping pulsar topics to database tables.

consistencyLevel

Query consistency level.

Default: LOCAL_ONE

DSE settings are:

For more, see How are consistent read and write operations handled?

nullToUnset

Whether to treat null values in Pulsar as UNSET in DSE.

Default: true

This parameter controls the handling of updates versus overrides.

When true (default), the DataStax Pulsar Connector treats null values as unset fields. It only updates fields that are not null when a new record arrives. DataStax recommends using the default to avoid creating unnecessary tombstones.

If false, fields with null values are visible in the DSE target table.

ttl

Time-to-live.

Default: -1 (disabled)

Set to the number of seconds before the data is automatically deleted from the DSE table. When you configure topic.<topic-name>.<keyspace-name>.<table-name>.ttl, all rows for that topic table will have this same TTL value. DataStax Pulsar Connector appends AND TTL <configured-ttl-value> to the INSERT statement for those rows.

deletesEnabled

Default: true

When enabled, treat records that after mapping would result in only non-null values for primary key columns as deletes, rather than inserting/updating nulls for all regular columns.

Only triggers if the setting is enabled and the mapping specification contains all DSE table columns.