Configure error handling

Specifies whether Kafka Connector should continue processing records even if an error occurred.

dse-sink-distributed.json.sample

  • The dse-sink-distributed.json.sample file is located in the conf directory of the DataStax Apache Kafka Connector distribution package.

dse-sink-standalone.properties.sample

  • The dse-sink-standalone.properties.sample file is located in the conf directory of the DataStax Apache Kafka Connector distribution package.

DataStax Apache Kafka™ Connector provides an optional parameter, ignoreErrors. When set to true, it allows the connector to continue processing records even after an error occurred on the prior record.

ignoreErrors
When ignoreErrors is false (default behavior), if one of your records makes your Kafka Connector fail, it prevents the process from progressing to the next record. The erroneous record is retried indefinitely and is written to the log each time. However, if you set ignoreErrors to true, any erroneous record is logged once and the next record is picked up by the Kafka Connector automatically.
If you decide to use ignoreErrors, there are two ways to deploy the Kafka Connector:
  1. In standalone mode for development or test environments only, you can specify ignoreErrors=true in your customized version of the dse-sink-standalone.properties.sample file.
  2. In distributed mode for production environments, you can specify "ignoreErrors": true in your customized version of the dse-sink-distributed.json.sample JSON file.
Because the Kafka Connector runs in the Kafka Connect Worker JVM, all of the logging entries go to the Kafka Connect Worker logs. For example, to pipe the worker process to the background and write the log entries to worker.log:
./connect-distributed.sh connect-distributed.properties &> worker.log &

Default: false.