Mapping messages to table that has a User Defined Type

The DataStax Apache Pulsar™ Connector processes JSON, Avro, and complex types directly into a user-defined type column in the supported database. The Pulsar message embedded field names and the UDT field names must match.

In the example stocks_topic, the key is a basic string and the value is regular JSON.

key value

APPLE

{"symbol":"APPL", "value":208, "exchange":"NASDAQ", "industry":"TECH", "ts":"2018-11-26T19:26:27.483"}

EXXON MOBIL

{"symbol":"M",
"value":80,
"exchange":"NYSE",
"industry":"ENERGY",
"ts":"2018-11-26T19:26:27.483"}

GENERAL MOTORS

{"symbol":"GM", "value":38, "exchange":"NYSE", "industry":"AUTO", "ts":"2018-11-26T19:26:27.483"}

AT&T

{"symbol":"AT&T",
"value":33,
"exchange":"NYSE",
"industry":"TELECOM",
"ts":"2018-11-26T19:26:27.483"}

FORD MOTOR

{"symbol":"F", "value":10, "exchange":"NYSE", "industry":"AUTO", "ts":"2018-11-26T19:26:27.483"}

The DataStax keyspace name is stocks_keyspace. In the stocks keyspace create the user defined type, stocks_type:

CREATE TYPE stocks_keyspace.stocks_type (
    symbol text,
    ts timestamp,
    exchange text,
    industry text,
    value double);

Define a table named stocks_udt_table that has a column for the primitive string field name and a second column that uses the type, stocks_type:

CREATE TABLE stocks_keyspace.stocks_table (
    name text primary key,
    stocks FROZEN<stocks_type>);

Configure the connector settings and use the following for the map specification:

topic:
    stocks_topic:
        stocks_keyspace:
            stocks_udt_table:
                mapping: 'name=key,stocks=value'