NodeSync metric MBeans

Use to track data validation operations.

The NodeSync MBeans track the data validation operations on a node.

Use DataStax Enterprise (DSE) OpsCenter to monitor these metrics in alerts and dashboard graphs.

Use the following syntax to locate and use NodeSync MBeans:
  • Global metric name syntax:
    com.datastax.nodesync:type=NodeSyncMetrics,name=metric_name
  • Per table metric name syntax:
    org.apache.cassandra.metrics:type=Table,keyspace=keyspace_name,scope=table_name,name=metric_name
The following screen shows the Global MBeans using jconsole:

NodeSync metrics

Object metrics

NodeSync repairs data and objects, the latter of which can be rows, range tombstones, and deletion information.

NodeSyncObjectsRepaired
Inconsistent objects (rows + range tombstones) that NodeSync validated and repaired. The number of rows and range tombstones is equivalent to NodeSyncDataRepaired.
NodeSyncObjectsValidated
Number of objects (rows + range tombstones) that NodeSync validated by checking for inconsistencies. The number of rows and range tombstones is equivalent to NodeSyncDataValidated.
NodeSyncRepairObjectsSent
Number of objects sent required to repair replicas. This attribute differs from NodeSyncDataRepaired in that 100 bytes of data that was inconsistent and repaired can be sent to multiple replicas, providing a more accurate representation of repair traffic generated between nodes.

Data metrics

NodeSyncDataValidated
Amount of data (in bytes) that NodeSync validated by checking for inconsistencies.
NodeSyncDataRepaired
Inconsistent data that NodeSync validated and repaired.
NodeSyncRepairDataSent
Size of data (in bytes) sent for repairing inconsistencies found by NodeSync. This metric differs from NodeSyncDataRepaired in that if a row of a specific size is inconsistent, that number is counted in NodeSyncDataRepaired. However, more or less of the same number can be counted in NodeSyncRepairDataSent, because the row can be inconsistent on more than one node, but only a portion of the row might be inconsistent (and thus sent).

Page metrics

NodeSync validtes data by pages, which are the least granular objects that NodeSync operates on. NodeSync collects pages by outcomes, and the resulting numbers are meaningful in comparison. For example, the percentage of pages already fully in sync that did not require repair.

NodeSyncFailedPages
Number of pages not validated due to an unexpected error.
NodeSyncFullInSyncPages
Number of pages fully validated (all replicas responded) that had no inconsistencies.
NodeSyncFullRepairedPages
Number of pages fully validated (all replicas responded) that had inconsistencies, and which were successfully repaired.
NodeSyncPartialInSyncPages
Number of pages partially validated (some replicas responded) that had no inconsistencies.
NodeSyncPartialRepairedPages
Number of pages partially validated (some replicas responded) that had inconsistencies, and which were successfully repaired.
NodeSyncProcessedPages
Total number of processed pages since the last node restart, calculated as the sum of all other page counters.
NodeSyncUncompletedPages
Subset of NodeSyncProcessedPages that failed to process. The reason could be that only one node was up and the page was skipped, some replicas failed during processing, or some unexpected error occurred.

NodeSync MBean attributes

Each of the following metrics can return some or all of the NodeSync MBean attributes.
MeanRate
Mean of the OneMinuteRate, FiveMinuteRate, and FifteenMinuteRate attributes.
OneMinuteRate
Number of objects processed for the given MBean in a one minute period.
FiveMinuteRate
Number of objects processed for the given MBean in a five minute period.
FifteenMinuteRate
Number of objects processed for the given MBean in a fifteen minute period.
RateUnit
Unit of measurement for rate processing, which is number of events per second.
Count
Total number of objects processed for the given MBean.

Example

To return the global mean rate of the node:
nodetool sjk mx -b "com.datastax.nodesync:type=NodeSyncMetrics,name=NodeSyncObjectsValidated" -f MeanRate -mg
The results show the object name and value.
com.datastax.nodesync:type=NodeSyncMetrics,name=NodeSyncObjectsValidated
2.542410005904823