Scaling down by removing workers

Decrease throughput by running the DataStax Connector on fewer workers.

To remove workers from a Kafka Connect Worker group, simply stop the worker. The Kafka Connect framework automatically scales down and rebalances the tasks amongst the alive workers.

Tip: The DataStax Apache Kafka Connector framework automatically rebalances the load when workers are added by reallocating the tasks among the workers.

Procedure

  1. Find the PID of the worker process. 
    ps auwx | grep ConnectDistributed
    Example out:
    automat+ 20689  9.6 10.3 4635620 846012 pts/2  Sl   00:36   0:16 /usr/lib/jvm/jdk1.8.0_151/bin/java 
...
org.apache.kafka.connect.cli.ConnectDistributed kafka/config/connect-distributed.properties
  2. Kill the process. 
    kill 20689