Mapping pulsar topics to database tables

DataStax Apache Pulsar™ Connector has a simple yet powerful syntax for mapping fields from a Pulsar record to columns in a supported database table. An instance of the DataStax Apache Pulsar™ Connector can capture multiple topics and supports mapping a single topic to one or more tables.

DataStax Apache Pulsar™ Connector topic mapping

DataStax Apache Pulsar™ Connector supports topic-to-table mappings with supported databases:

Determining topic data structure Display messages to determine the data structure of the topic messages.
Mapping basic messages to table columns Create a topic-table map for Pulsar messages that only contain a key and value in each record.
Mapping a message that contain JSON fields For JSON fields, map individual fields in the structure to columns.
Mapping Avro messages Supports mapping individual fields from a Avro format field.
Extract Pulsar record header values Extract values from Pulsar record header and write to the database table.
Mapping messages to table that has a User Defined Type Write complex types directly into User-defined Types (UDT).
Mapping a topic to multiple tables Ingest a single topic into multiple tables using a single connector instance.
Multiple topics to multiple tables Ingest multiple topics and write to different tables using a single connector instance.
Selectively update maps and UDTs based on Pulsar fields Selectively update maps and UDTs based on Pulsar fields.
Provide CQL queries in mappings Provide CQL queries when new record arrives on the Pulsar topic.
The now() function in mappings You can use the now() function in mappings.