Pulsar topic-to-table settings

Capture Apache Pulsar™ topics in the supported database by specifying a target keyspace, table, and then map the topic fields to table columns in the name of the parameter.

Parameters

Use the following syntax for standalone properties file:

topics: topic_list
topic:
  topic_name:
    keyspace_name:
      table_name:
        mapping: mapping_specification
        consistencyLevel: WRITE_CONSISTENCY_LEVEL
        ttlTimeUnit: seconds
        ttl: -1
        nullToUnset: true
        deletesEnabled: true

topics

A comma separated list of all topics to which the DataStax Connector subscribes.

where:

topic_name - Pulsar topic name.
keyspace_name - database keyspace where the table is located.
table_name - database table where data is written.

mapping: Required, field-to-column mapping. See Mapping pulsar topics to database tables.

consistencyLevel

Query consistency level. DSE settings are:

ALL
EACH QUORUM
QUORUM
LOCAL_QUORUM
ONE
TWO
THREE
LOCAL\_ONE (default)
ANY

See How are consistent read and write operations handled?

Default: LOCAL_ONE

nullToUnset: Whether to treat nulls in Pulsar as UNSET in DSE. DataStax recommends using the default to avoid creating unnecessary tombstones.

Default: true

ttl: Time-to-live. Set to the number of seconds before the data is automatically deleted from the DSE table. When you configure topic.<topic-name>.<keyspace-name>.<table-name>.ttl, all rows for that topic table will have this same TTL value. DataStax Pulsar Connector appends AND TTL <configured-ttl-value> to the INSERT statement for those rows.

Default: -1 (disabled)

deletesEnabled

When enabled, treat records that after mapping would result in only non-null values for primary key columns as deletes, rather than inserting/updating nulls for all regular columns.

Only triggers if the setting is enabled and the mapping specification contains all DSE table columns.

Default: true

Was this helpful?

Edit this page