DataStax Apache Kafka Connector release notes

Release notes for DataStax Apache Kafka Connector.

Release notes for DataStax Apache Kafka™ Connector.

DataStax Apache Kafka™ Connector is installed in the Kafka Connect framework and synchronizes records from a Kafka topic to table rows in the following supported databases:

DataStax Enterprise (DSE) 4.7 and later databases
Open source Apache Cassandra™ 2.1 and later databases
DataStax Distribution of Apache Cassandra (DDAC) databases
DataStax Apollo cloud databases

For related information, refer to the DataStax Enterprise release notes:

DataStax Apache Kafka Connector 1.2.1 release notes

16 December 2019

DataStax Apache Kafka™ Connector 1.2.1 release notes

1.2.1 Changes and enhancements

This release, DataStax Apache Kafka Connector 1.2.1, adds support for topic-to-table mappings with table rows of open source Apache Cassandra databases. (KAF-165)

The new support in 1.2.1 is in addition to the existing functionality for topic-to-table mappings with:

DataStax Enterprise (DSE) 4.7 and later databases
DataStax Distribution of Apache Cassandra (DDAC) databases
DataStax Apollo cloud databases

DataStax Apache Kafka Connector 1.2.0 release notes

12 November 2019

DataStax Apache Kafka™ Connector 1.2.0 release notes

1.2.0 Changes and enhancements

You can deploy DataStax Apache Kafka Connector to stream records from an Apache Kafka topic to a cloud-based DataStax Apollo database by using the cloud.secureConnectBundle. See Streaming data with the DataStax Apache Kafka Connector. (KAF-143)
Note: DataStax Apollo Open Beta participants can download the secure connect bundle from the DataStax Constellation console after creating an Apollo database.
You can pass all DataStax Apache Kafka Connector settings to the DataStax Java driver directly by using the datastax-java-driver prefix. See Pass Kafka Connector settings directly to the DataStax Java driver (KAF-79)
In the DataStax Apache Kafka Connector topic mapping, you can optionally extract a value from the record's header by using header.header-field-name. See Extract Kafka record header values. (KAF-142)

DataStax Apache Kafka Connector 1.1.1 release notes

23 September 2019

DataStax Apache Kafka Connector 1.1.1 release notes include:

1.1.1 Changes and enhancements

This release adds a new parameter, ignoreErrors. When set to true, it allows the Kafka Connector to continue processing records even after an error occurred on the prior record. Refer to Configure error handling. (KAF-132)
DataStax Apache Kafka Connector has been upgraded to use the latest:
- DataStax Enterprise Java Driver 2.2.0
- Apache Cassandra OSS Driver 4.2.0
(KAF-136)

1.1.1 Resolved issues

Bootstrapping node results in task failure and NPE. (KAF-126)
The dse-reference.conf file not loaded because wrong config Loader used. (KAF-135)

DataStax Apache Kafka Connector 1.1.0 release notes

20 May 2019

DataStax Apache Kafka Connector 1.1.0 release notes include:

1.1.0 Changes and enhancements

New mapping properties:
- __timestamp. By setting this property you can specify which column should be used as the write-time timestamp when doing an insert to DSE or DDAC. Refer to Specify writetime timestamp column (KAF-46)
- __ttl. By setting this property you can specify which column is used as the TTL (Time-to-Live) when doing an insert to DSE or DDAC. Refer to Setting row-level TTL values from Kafka fields. (KAF-107)
Counters and histograms are now applicable per topic, keyspace, and table. Previously counters and histograms were implemented at a global level. The change allows for finer granularity in reporting operations. As a result, the name parameter value in objectName has changed.
For recordCount:
- From: objectName='com.datastax.kafkaconnector:connector=*,name=recordCount'
- To: objectName='com.datastax.kafkaconnector:connector=*,name=topic.keyspace.table.recordCount'
For failedRecordCount:
- From: objectName='com.datastax.kafkaconnector:connector=*,name=failedRecordCount'
- To: objectName='com.datastax.kafkaconnector:connector=*,name=topic.keyspace.table.failedRecordCount'
Refer to Metrics for the processed Kafka topic records and Failed Kafka topic record count. (KAF-72).
Support for the Confluent Kafka Connect 2.1.0 and 2.2.0 API. (KAF-81, KAF-109).
Kafka topic names may now include one or more period (.) characters. Example: org.datastax.init.event.history (KAF-104)
In cases where a Kafka producer writes a record's entire value as null, DataStax Apache Kafka Connector now allows the DSE or DDAC record to be deleted, provided:
- topic.<topic>.<keyspace>.<table>.deletesEnabled": "true" is set in the configuration
- The record's primary key and clustering key columns are present for the event
Given those conditions, the DSE or DDAC rows can be deleted for the primary key in this example:
```
{"key":{"symbol":"DEC", "industry": "tech"}, "value": null}
```
Note: The connector continues to support the scenario where the DSE or DDAC DELETE can be performed when all field values (other than the primary key and clustering keys) are null.
(KAF-113)
To allow for the recording of latency records, DataStax Apache Kafka Connector dynamically sets the default value of advanced.metrics.session.cql-requests.highest-latency so that it exceeds the configured request-timeout value. (KAF-115)
DataStax Apache Kafka Connector has been upgraded to use the externally released version of the DSE Java Driver 2.x. (KAF-112)

1.1.0 Resolved issues

java.lang.ArrayIndexOutOfBoundsException while inserting (KAF-114)

DataStax Apache Kafka Connector 1.0 release notes

5 December 2018

The latest version of DataStax Apache Kafka Connector is 1.2.1.

DataStax Apache Kafka Connector 1.0 includes these features and enhancements:

JSON, Avro, Struct, Primitive Types mapping support. (KAF-1, KAF-3, KAF-7)
DSE data types support. (KAF-4)
Configurable row TTL. (KAF-6)
Configurable consistency level. (KAF-9)
Treat nulls as unset. (KAF-11)
Support multiple topics for single connector instance. (KAF-14)
Report metrics via JMX. (KAF-15)
Configurable max request rate with maxConcurrentRequests. (KAF-16)
Support for connections to:
- DataStax Enterprise (DSE) 5.0 and later databases
- DataStax Distribution of Apache Cassandra™ 3.11 (DDAC) databases
Note: All nodes in each cluster must be uniformly licensed to use the same subscription. For example, if a cluster contains 5 nodes, all 5 must be DDAC nodes, or all 5 must be DSE nodes.
Connector to DSE and DDAC SSL. (KAF-18)
Connector to DSE and DDAC username/password authentication. (KAF-19)
Connector to DSE and DDAC Kerberos authentication. (KAF-20)
Configurable deletes. (KAF-21)
Configurable date/time formats. (KAF-26)
Mapping single topic to multiple DSE and DDAC tables. (KAF-43)
Connector to DSE and DDAC compression. (KAF-45)
Configurable connector to DSE and DDAC execution timeout with queryExecutionTimeout. (KAF-49)
Configurable max statements per batch with maxNumberOfRecordsInBatch. (KAF-60)
Configurable connections per DSE host with connectionPoolLocalSize. (KAF-95)