Mapping messages to table that has a User Defined Type

Write complex types directly into User-defined Types (UDT).

The DataStax Apache Kafka Connector processes JSON, Avro, and complex types, such as Kafka Structs or maps, directly into a user-defined type column in the supported database. The Kafka message embedded field names and the UDT field names must match.

In the example stocks_topic, the key is a basic string and the value is regular JSON.


key	value
APPLE	`{"symbol":"APPL", "value":208, "exchange":"NASDAQ", "industry":"TECH", "ts":"2018-11-26T19:26:27.483"}`
EXXON MOBIL	`{"symbol":"M", "value":80, "exchange":"NYSE", "industry":"ENERGY", "ts":"2018-11-26T19:26:27.483"}`
GENERAL MOTORS	`{"symbol":"GM", "value":38, "exchange":"NYSE", "industry":"AUTO", "ts":"2018-11-26T19:26:27.483"}`
AT&T	`{"symbol":"AT&T", "value":33, "exchange":"NYSE", "industry":"TELECOM", "ts":"2018-11-26T19:26:27.483"}`
FORD MOTOR	`{"symbol":"F", "value":10, "exchange":"NYSE", "industry":"AUTO", "ts":"2018-11-26T19:26:27.483"}`

The DataStax keyspace name is stocks_keyspace. In the stocks keyspace create the user defined type, stocks_type:

CREATE TYPE stocks_keyspace.stocks_type (
    symbol text, 
    ts timestamp, 
    exchange text,
    industry text, 
    value double);

Define a table named stocks_udt_table that has a column for the primitive string field name and a second column that uses the type, stocks_type:

CREATE TABLE stocks_keyspace.stocks_table (
    name text primary key, 
    stocks FROZEN<stocks_type>);

Configure the connector settings and use the following for the map specification:

“topic.stocks_topic.stocks_keyspace.stocks_udt_table.mapping”: 
    “name=key,stocks=value”