Scaling down by removing workers

Decrease throughput by running the DataStax Connector on fewer workers.

To remove workers from a Kafka Connect Worker group, simply stop the worker. The Kafka Connect framework automatically scales down and rebalances the tasks amongst the alive workers.

Tip: The Kafka Connect framework automatically rebalances the load when workers are added by reallocating the tasks among the workers.

Procedure

  1. Find the PID of the worker process.
    ps auwx | grep ConnectDistributed
    Example out:
    automat+ 20689  9.6 10.3 4635620 846012 pts/2  Sl   00:36   0:16 /usr/lib/jvm/jdk1.8.0_151/bin/java 
    ...
    org.apache.kafka.connect.cli.ConnectDistributed kafka/config/connect-distributed.properties
  2. Kill the process.
    kill 20689