DSE Advanced Replication metrics
DSE Advanced Replication metrics on the source nodes refer to the current status of that node in the source cluster.
cassandra-env.sh
The location of the cassandra-env.sh file depends on the type of installation:Package installations | /etc/dse/cassandra/cassandra-env.sh |
Tarball installations | installation_location/resources/cassandra/conf/cassandra-env.sh |
Collect metrics on each source node to review the current status of that node in the source cluster. A working source and destination configuration is required to use the metrics feature. See Getting started.
Ensure JMX access
Metrics are stored in the DataStax Enterprise JMX system. JMX access is required.
- For production, DataStax recommends authenticating JMX users, see .
- Use these steps to enable local JMX access. Localhost access is useful for test and development.
- On the source node, edit cassandra-env.sh and enable
local
JMX:
JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=localhost" LOCAL_JMX=yes
- On the source node, stop and restart DataStax Enterprise to recognize the local JMX change.
Display metrics on the command line
Use the
dse advrep
command line tool to display metrics on the command
line. Ensure that the source node meets the command line prerequisites. - On the source
node:
dse advrep --jmx-port 7199 metrics list
------------------------------------------ |Group |Type |Count| ------------------------------------------ |Tables |MessagesDelivered |1002 | ------------------------------------------ |ReplicationLog|CommitLogsToConsume|1 | ------------------------------------------ |Tables |MessagesReceived |1002 | ------------------------------------------ |ReplicationLog|MessageAddErrors |0 | ------------------------------------------ |ReplicationLog|CommitLogsDeleted |0 | ------------------------------------------ ------------------------------------------------------------------------------------------------------------------------------------------------- |Group |Type |Count|RateUnit |MeanRate |FifteenMinuteRate |OneMinuteRate |FiveMinuteRate | ------------------------------------------------------------------------------------------------------------------------------------------------- |ReplicationLog|MessagesAdded |1002 |events/second|0.012688461014514603|9.862886141388435E-39|2.964393875E-314 |2.322135514219019E-114 | ------------------------------------------------------------------------------------------------------------------------------------------------- |ReplicationLog|MessagesDeleted |0 |events/second|0.0 |0.0 |0.0 |0.0 | ------------------------------------------------------------------------------------------------------------------------------------------------- |ReplicationLog|MessagesAcknowledged |1002 |events/second|0.012688456391385135|9.86403600116801E-39 |2.964393875E-314 |2.3230339468969963E-114| ------------------------------------------------------------------------------------------------------------------------------------------------- |ReplicationLog|CommitLogMessagesRead|16873|events/second|0.21366497971804438 |0.20580430240786005 |0.39126032533612265|0.2277227124698431 | ------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------- |Group |Type |Value| ------------------------------------- |Transmission|AvailablePermits|30000| -------------------------------------
Accessing the metrics
Use JMX to access the metrics. Any JMX tool, such as
jconsole
, can access
the MBeans for advanced replication. The port listed above, 7199
, is used
with the hostname or IP address:Choose the MBeans tab and find com.datastax.bdp.advrep.v2.metrics
in
the left-hand navigation frame:The example shown here displays the attributes for
com.datastax.bdp.advrep.v2.metrics:type=ReplicationLog,name=MessagesAdded
.Performance metrics
Metrics are exposed as JMX
MBeans under the
Metrics are available for the following groups:Metrics are also available per table:Descriptions of each metric is provided.com.datastax.bdp.advrep.v2.metrics
path and are logically
divided into main groups. Each group refers to an architecture component. Metrics types are:- Counter
- A simple incrementing and decrementing 64-bit integer.
- Meter
- Measures the rate at which a set of events occur.
- Histogram
- Measures the distribution of values in a stream of data.
- Timer
- A histogram of the duration of a type of event and a meter of the rate of its occurrence.
- Gauge
- A gauge is an instantaneous measurement of a value.
Note: Metrics for DSE 5.0 (V1) are still present;
see the DSE 5.0 documentation for those
metrics.
ReplicationLog
Metrics for the ReplicationLog group:
Metric name | Description | Metric type |
---|---|---|
MessagesAdded | The number of messages that were added to the replication log, and the rate that the messages were added, per replica. | Meter |
MessagesAcknowledged | The number of messages that were acknowledged (and removed) from the replication log. Acknowledgement can be 1 or 1+n if errors occur. | Meter |
MessagesDeleted | The number of messages that were deleted from the replication log, including invalid messages and messages that were removed after a channel truncate operation. | Meter |
MessageAddErrors | The number of errors that occurred when adding a message to the replication log. | Counter |
CommitLogsToConsume | The number of commit logs that need to be consumed that have advanced replication messages. | Counter |
CommitLogMessagesRead | The number of commit log messages added to the replication log. The commit log messages are read if a message pertains to a source table that has collection enabled. | Meter |
CommitLogMessagesDeleted | The number of commit log messages deleted from the commit log after adding to the replication log. Like CommitLogMessagesRead, this metric only pertains to messages in tables that are enabled for advanced replication. | Meter |
Transmission
Metrics for the Transmission group:
Metric name | Description | Metric type |
---|---|---|
AvailablePermits | The current number of available global permits for transmission. | Gauge |
AdvancedReplicationHub-[destinationName]-metrics
Metrics for the AdvancedReplicationHub-[destinationName]-metrics group are provided
automatically by the DSE Java driver. Incomplete examples of per-destination-metrics are:
For details, see the DSE Java driver documentation.
Metric name | Metric type |
---|---|
known-hosts | Counter |
connected-to | Counter |
open-connections | Counter |
requests-timer | Timer |
connection-errors | Counter |
write-timeouts | Counter |
read-timeouts | Counter |
unavailables | Counter |
other-errors | Counter |
retries | Counter |
ignores | Counter |
Performance metrics per table
Use JMX to find performance metrics per table, look under the
For example, to access the com.datastax.bdp.advrep.v2.metrics
tab in the left-hand navigation frame
for Tables
, select a table and inspect the metrics:MessagesReceived
metric for the table
sensor_readings
in the keyspace demo
look at the following
path:com.datastax.bdp.advrep.v2.metrics:type=Tables,scope=demo.sensor_readings,name=MessagesReceived
The
following metrics are provided per table:
Metric name | Description | Metric type |
---|---|---|
MessagesReceived | The number of messages received from the source cluster for this table. | Counter |
MessagesDelivered | The number of messages for the source table that were replicated to the destination. | Counter |
MessagesDeleted | The number of messages that were deleted from the replication log, including invalid messages and messages that were removed after a channel truncate operation. | Counter |