DataStax Apache Kafka Connector - Batch size metrics

Write statistics for each mapping of a Kafka topic to a DataStax table.

The DataStax Apache Kafka™ Connector records metrics for each Kafka topic that is synchronized with a DataStax database table. The metric corresponds to a topic mapping. See Mapping kafka topics to DataStax Enterprise database tables.

The path to the MBeans for the topic-table map is:
objectName='com.datastax.kafkaconnector:connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name,name=*'
where the variables above correspond to settings in the connector configuration file:
  • connector_name - DataStax Apache Kafka™ Connector name.
  • topic_name is Kafka prefix.
  • keyspace_name is the case-sensitive DataStax database keyspace name.
  • table_name is the case-sensitive DataStax database table name.

Batch size

Number of statements in the CQL batch used to write records to the DataStax database. The connector default limit is 32. See maxNumberOfRecordsInBatch.

If the limit is consistently hit, then increasing the value can result in better connector throughput if the database nodes can handle the added pressure of larger CQL batch statements. When increasing maxNumberOfRecordsInBatch, ensure that the batch size does not exceed the maximum allowed by batch_size_fail_threshold_in_kb. See Writing fails because of mutation size for details.

The path the batch size MBean is:
objectName='com.datastax.kafkaconnector:connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name
where the variables above correspond to settings in the connector configuration file:
  • connector_name - DataStax Apache Kafka™ Connector name.
  • topic_name is Kafka prefix.
  • keyspace_name is the case-sensitive DataStax database keyspace name.
  • table_name is the case-sensitive DataStax database table name.
The attributes are:
Mean
The average number of write requests per batch statement.
objectName='com.datastax.kafkaconnector:connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name,name=batchSize' attribute='Mean'
Count
Running total of all batch sizes.
objectName='com.datastax.kafkaconnector:connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name,name=batchSize' attribute='Count'
50thPercentile
objectName='com.datastax.kafkaconnector:connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name,name=batchSize' attribute='50thPercentile'
75thPercentile
objectName='com.datastax.kafkaconnector:connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_namename=batchSize' attribute='75thPercentile'
95thPercentile
objectName='com.datastax.kafkaconnector:connector=connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name,name=batchSize' attribute='95thPercentile'
98thPercentile
objectName='com.datastax.kafkaconnector:connector=connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name,name=batchSize' attribute='98thPercentile'
99thPercentile
objectName='com.datastax.kafkaconnector:connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name,name=batchSize' attribute='99thPercentile'
999thPercentile
objectName='com.datastax.kafkaconnector:connector=connector=connector_name,topic=topic_name,keyspace=keyspace_name,table=table_name,name=batchSize' attribute='999thPercentile'