DataStax Apache Pulsar™ connector metrics

You can view DataStax Apache Pulsar™ connector metrics in raw Prometheus format at the Apache Pulsar metrics endpoint: http://PULSAR_INSTANCE:8080/metrics.

You can also configure a Prometheus instance to monitor your Pulsar instance.

Prometheus query roots and attributes

The entries in this reference are formatted as Prometheus query strings.

The root of each query is a combination of com_datastax_oss_pulsar_sink, sink name, and metric class. The sink name refers to a configured and running DataStax Pulsar connector sink, and metric class is the general class of the metric, such as Count or SnapShotSize. For example, a complete query root could be com_datastax_oss_pulsar_sink_dse_sink_kv_Count.

In addition to the Prometheus query root, you can append a name attribute to focus on specific topics. For example, to focus on the failedRecordCount slice of the Count metric class, you can use the attribute name="failedRecordCount":

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{name="failedRecordCount"}

The following example shows a complete Prometheus query that includes the root, name attribute, and other attributes. Including the name attribute causes the response to include only the number of failed records (25).

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="failedRecordCount", table="pulsar_kv", topic="example_topic"}
25

This reference is organized by root query. Each section includes the available metrics for the name attribute along with an example query.

Specifying only the root query returns all of the available named metrics.

com_datastax_oss_pulsar_sink_<sink_name>_Count

Integer count for various DataStax Pulsar Connect metrics.

cql-client-timeouts

Number of CQL client timeouts.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{driver="driver", instance="localhost:8080", job="pulsar", name="cql-client-timeouts", session="s0"}

cql-requests

Number of CQL requests.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{driver="driver", instance="localhost:8080", job="pulsar", name="cql-requests", session="s0"}

batchSize

Returns the number of statements in the CQL batch used to write records to the database.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSize", table="pulsar_kv", topic="example_topic"}

batchSizeInBytes

Calculated size of every batch statement in bytes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSizeInBytes", table="pulsar_kv", topic="example_topic"}

failedRecordCount

Running total of the number of failed records encountered by the connector instance.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="failedRecordCount", table="pulsar_kv", topic="example_topic"}

recordCount

Running total of records processed.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="recordCount", table="pulsar_kv", topic="example_topic"}

failedRecordsWithUnknownTopic

Number of failed record transactions with an unknown topic.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Count{instance="localhost:8080", job="pulsar", name="failedRecordsWithUnknownTopic"}

com_datastax_oss_pulsar_sink_<sink_name>_SnapshotSize

Snapshot size.

batchSize

Returns the number of statements in the CQL batch used to write records to the database.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_SnapshotSize{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSize", table="pulsar_kv", topic="example_topic"}

batchSizeInBytes

Calculated size of every batch statement in bytes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_SnapshotSize{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSizeInBytes", table="pulsar_kv", topic="example_topic"}

com_datastax_oss_pulsar_sink_<sink_name>_<n>thPercentile

Percentile latency histograms where <n> is one of 999, 99, 98, 95, 75, or 50. For example com_datastax_oss_pulsar_sink_dse_sink_kv_95thPercentile returns metrics for the 95th percentile latency histogram.

cql-requests

Number of CQL requests in the given percentile latency histogram.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_99thPercentile{driver="driver", instance="localhost:8080", job="pulsar", name="cql-requests", session="s0"}

batchSize

Returns the number of statements in the CQL batch used to write records to the database in the given percentile latency histogram.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_95thPercentile{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSize", table="pulsar_kv", topic="example_topic"}

batchSizeInBytes

Calculated size of every batch statement in bytes in the given percentile latency histogram.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_50thPercentile{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSizeInBytes", table="pulsar_kv", topic="example_topic"}

com_datastax_oss_pulsar_sink_<sink_name>_<n>MinuteRate

Moving average of records processed in the last <n> minutes where <n> is one of One, Five, or Fifteen. For example, com_datastax_oss_pulsar_sink_dse_sink_kv_OneMinuteRate returns metrics for the one-minute moving average.

cql-requests

Number of CQL requests in the last one, five, or fifteen minutes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_OneMinuteRate{driver="driver", instance="localhost:8080", job="pulsar", name="cql-requests", session="s0"}

recordCount

Moving average of records in the last one, five, or fifteen minutes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_FifteenMinuteRate{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="recordCount", table="pulsar_kv", topic="example_topic"}

failedRecordCount

Moving average of failed records in the last one, five, or fifteen minutes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_FiveMinuteRate{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="failedRecordCount", table="pulsar_kv", topic="example_topic"}

failedRecordsWithUnknownTopic

Moving average of failed records with unknown topics processed in the last five or fifteen minutes. Not available for OneMinuteRate.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_FiveMinuteRate{instance="localhost:8080", job="pulsar", name="failedRecordsWithUnknownTopic"}

com_datastax_oss_pulsar_sink_<sink_name>_Max

Maximum number of requests.

cql-requests

Maximum number of CQL requests.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Max{driver="driver", instance="localhost:8080", job="pulsar", name="cql-requests", session="s0"}

batchSize

Returns the number of statements in the CQL batch used to write records to the database.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Max{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSize", table="pulsar_kv", topic="example_topic"}

batchSizeInBytes

Calculated size of every batch statement in bytes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Max{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSizeInBytes", table="pulsar_kv", topic="example_topic"}

com_datastax_oss_pulsar_sink_<sink_name>_Min

Minimum number of records processed.

cql-requests

Minimum number of CQL requests processed.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Min{driver="driver", instance="localhost:8080", job="pulsar", name="cql-requests", session="s0"}

batchSize

Returns the number of statements in the CQL batch used to write records to the database.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Min{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSize", table="pulsar_kv", topic="example_topic"}

batchSizeInBytes

Calculated size of every batch statement in bytes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Min{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSizeInBytes", table="pulsar_kv", topic="example_topic"}

com_datastax_oss_pulsar_sink_<sink_name>_Mean

Mean number of requests processed.

cql-requests

Mean number of CQL requests processed.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Mean{driver="driver", instance="localhost:8080", job="pulsar", name="cql-requests", session="s0"}

batchSize

Returns the number of statements in the CQL batch used to write records to the database.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Mean{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSize", table="pulsar_kv", topic="example_topic"}

batchSizeInBytes

Calculated size of every batch statement in bytes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_Mean{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSizeInBytes", table="pulsar_kv", topic="example_topic"}

com_datastax_oss_pulsar_sink_<sink_name>_MeanRate

The connector’s lifetime average number of records processed.

cql-requests

Average number of CQL requests processed since the connector was created.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_MeanRate{driver="driver", instance="localhost:8080", job="pulsar", name="cql-requests", session="s0"}

recordCount

Average number of records processed since the connector was created.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_MeanRate{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="recordCount", table="pulsar_kv", topic="example_topic"}

failedRecordCount

Average number of failed records processed since the connector was created.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_MeanRate{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="failedRecordCount", table="pulsar_kv", topic="example_topic"}

failedRecordsWithUnknownTopic

Average number of failed records with unknown topics processed since the connector was created.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_MeanRate{instance="localhost:8080", job="pulsar", name="failedRecordsWithUnknownTopic"}

com_datastax_oss_pulsar_sink_<sink_name>_StdDev

Standard deviation.

cql-requests

Standard deviation for CQL requests.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_StdDev{driver="driver", instance="localhost:8080", job="pulsar", name="cql-requests", session="s0"}

batchSize

Returns the number of statements in the CQL batch used to write records to the database.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_StdDev{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSize", table="pulsar_kv", topic="example_topic"}

batchSizeInBytes

Calculated size of every batch statement in bytes.

Example

com_datastax_oss_pulsar_sink_dse_sink_kv_StdDev{instance="localhost:8080", job="pulsar", keyspace="pulsar_qs", name="batchSizeInBytes", table="pulsar_kv", topic="example_topic"}

DataStax Apache Pulsar™ connector metrics

Prometheus query roots and attributes

com_datastax_oss_pulsar_sink_<sink_name>_Count

com_datastax_oss_pulsar_sink_<sink_name>_SnapshotSize

com_datastax_oss_pulsar_sink_<sink_name>_<n>thPercentile

com_datastax_oss_pulsar_sink_<sink_name>_<n>MinuteRate

com_datastax_oss_pulsar_sink_<sink_name>_Max

com_datastax_oss_pulsar_sink_<sink_name>_Min

com_datastax_oss_pulsar_sink_<sink_name>_Mean

com_datastax_oss_pulsar_sink_<sink_name>_MeanRate

com_datastax_oss_pulsar_sink_<sink_name>_StdDev

Was this helpful?

Give Feedback