Installing the DataStax Kafka Connector 1.1.0

Install on Linux-based platform using a binary tarball.

Install the connector from the DataStax distribution binary tarball using an account that has write access to the Kafka configuration directory.

dse-sink-standalone.properties.sample

  • The dse-sink-standalone.properties.sample file is located in the conf directory of the DataStax Apache Kafka Connector distribution package.

dse-sink-distributed.json.sample

  • The dse-sink-distributed.json.sample file is located in the conf directory of the DataStax Apache Kafka Connector distribution package.

Supported Apache Kafka versions

Install the DataStax Apache Kafka™ Connector on any of following versions:
  • Confluent 3.3 and later
  • Apache Kafka 0.11 and later

Prerequisites

Procedure

Important: End User License Agreement (EULA). By downloading this DataStax product, you agree to the terms of the EULA.

Perform the following steps on a Kafka Connect node:

  1. Download the tarball from the DataStax downloads site. If you agree, enable the Terms checkbox and click the download icon.
  2. Extract the files:
    tar zxf kafka-connect-dse-version_number.tar.gz
    where version_number is the connector version number.
    The following files are unpacked into a directory such as kafka-connect-dse-version_number.
    LICENSE.txt
    README.md
    THIRD-PARTY.txt
    conf/dse-sink-distributed.json.sample
    conf/dse-sink-standalone.properties.sample
    kafka-connect-dse-version_number.jar
    where version_number is the DSE connector version number.
  3. Configured the DataStax connector JAR using one of the following methods:
    • Move the DataStax connector JAR to the Kafka plugins directory:
      mv installation_location/kafka-connect-dse-version_number.jar kafka_plugins_dir
    • Configure the path to the JAR:
      Apache Kafka 0.11.x and later - Specify the JAR location in the plugin.path parameter in the connect-standalone.properties or connect-distributed.properties file that is passed to the worker start-up scripts. Example:
      plugin.path=install_location/kafka-connect-dse-1.1.0.jar
      Note: Confluent 3.3 and later are supported.
  4. Copy the sample configuration file from kafka-connect-dse-version_number/conf/ to the Kafka configuration directory, which is typically the config or etc directory.
    DataStax provides the following sample files in the conf directory of the connector distribution package:
    • dse-sink-standalone.properties.sample for standalone mode. It is a Java properties file that contains all settings with descriptions. Settings with a default value are commented out.
    • dse-sink-distributed.json.sample for distributed mode. This file is in JSON format and contains all settings, which are enumerated and active. To use the default values, remove settings from the configuration file. JSON does not support comments.
  5. Rename the sample file to dse-sink.properties or dse-sink.json.
  6. Update the settings as necessary. See Configuring the DataStax Apache Kafka Connector.
  7. Ensure that the user running Kafka has permission to access the configuration and JAR files.