DataStax Apache Kafka Connector release notes

Release notes for DataStax Apache Kafka Connector.

Release notes for DataStax Apache Kafka™ Connector.

For DSE release notes, see DataStax Enterprise 6.7 Release Notes.

DataStax Apache Kafka Connector 1.1.0 release notes

20 May 2019

DataStax Apache Cassandra Connector 1.1.0 release notes include:

1.1.0 Changes and enhancements

  • New mapping properties:
    • __timestamp. By setting this property you can specify which column should be used as the write-time timestamp when doing an insert to DSE or DDAC. Refer to Specify writetime timestamp column (KAF-46)
    • __ttl. By setting this property you can specify which column is used as the TTL (Time-to-Live) when doing an insert to DSE or DDAC. Refer to Setting row-level TTL values from Kafka fields. (KAF-107)
  • Counters and histograms are now applicable per topic, keyspace, and table. Previously counters and histograms were implemented at a global level. The change allows for finer granularity in reporting operations. As a result, the name parameter value in objectName has changed.
    For recordCount:
    • From: objectName='com.datastax.kafkaconnector:connector=*,name=recordCount'
    • To: objectName='com.datastax.kafkaconnector:connector=*,name=topic.keyspace.table.recordCount'
    For failedRecordCount:
    • From: objectName='com.datastax.kafkaconnector:connector=*,name=failedRecordCount'
    • To: objectName='com.datastax.kafkaconnector:connector=*,name=topic.keyspace.table.failedRecordCount'
    Refer to Metrics for the processed Kafka topic records and Failed Kafka topic record count. (KAF-72).
  • Support for the Confluent Kafka Connect 2.1.0 and 2.2.0 API. (KAF-81, KAF-109).
  • Kafka topic names may now include one or more period (.) characters. Example: org.datastax.init.event.history (KAF-104)
  • In cases where a Kafka producer writes a record's entire value as null, DataStax Apache Kafka Connector now allows the DSE or DDAC record to be deleted, provided:
    • topic.<topic>.<keyspace>.<table>.deletesEnabled": "true" is set in the configuration
    • The record's primary key and clustering key columns are present for the event
    Given those conditions, the DSE or DDAC rows can be deleted for the primary key in this example: 
    {"key":{"symbol":"DEC", "industry": "tech"}, "value": null}
    Note: The connector continues to support the scenario where the DSE or DDAC DELETE can be performed when all field values (other than the primary key and clustering keys) are null.
    (KAF-113)
  • To allow for the recording of latency records, DataStax Apache Kafka Connector dynamically sets the default value of advanced.metrics.session.cql-requests.highest-latency so that it exceeds the configured request-timeout value. (KAF-115)
  • DataStax Apache Kafka Connector has been upgraded to use the externally released version of the DSE Java Driver 2.x. (KAF-112)

1.1.0 Resolved issues

  • java.lang.ArrayIndexOutOfBoundsException while inserting (KAF-114)

DataStax Apache Kafka Connector 1.0 release notes

5 December 2018

The latest version of DataStax Apache Kafka Connector is 1.1.0.

DataStax Apache Kafka Connector 1.0 includes these features and enhancements:
  • JSON, Avro, Struct, Primitive Types mapping support. (KAF-1, KAF-3, KAF-7)
  • DSE data types support. (KAF-4)
  • Configurable row TTL. (KAF-6)
  • Configurable consistency level. (KAF-9)
  • Treat nulls as unset. (KAF-11)
  • Support multiple topics for single connector instance. (KAF-14)
  • Report metrics via JMX. (KAF-15)
  • Configurable max request rate with maxConcurrentRequests. (KAF-16)
  • Support for connections to:
    • DataStax Enterprise (DSE) 5.0 and later databases
    • DataStax Distribution of Apache Cassandra™ 3.11 (DDAC) databases
    Note: All nodes in each cluster must be uniformly licensed to use the same subscription. For example, if a cluster contains 5 nodes, all 5 must be DDAC nodes, or all 5 must be DSE nodes.
  • Connector to DSE and DDAC SSL. (KAF-18)
  • Connector to DSE and DDAC username/password authentication. (KAF-19)
  • Connector to DSE and DDAC Kerberos authentication. (KAF-20)
  • Configurable deletes. (KAF-21)
  • Configurable date/time formats. (KAF-26)
  • Mapping single topic to multiple DSE and DDAC tables. (KAF-43)
  • Connector to DSE and DDAC compression. (KAF-45)
  • Configurable connector to DSE and DDAC execution timeout with queryExecutionTimeout. (KAF-49)
  • Configurable max statements per batch with maxNumberOfRecordsInBatch. (KAF-60)
  • Configurable connections per DSE host with connectionPoolLocalSize. (KAF-95)