OpsCenter Metrics Tooltips Reference

The number of write requests per second on the coordinator nodes, analogous to client writes. Monitoring the number of requests over a given time period reveals system write workload and usage patterns.

The min, median, max, 90th, and 99th percentiles of a client writes. The time period starts when a node receives a client write request, and ends when the node responds back to the client. Depending on consistency level and replication factor, this may include the network latency from writing to the replicas.

The number of write requests on the coordinator nodes that fail due to errors returned from replicas.

The number of server write timeouts per second on the coordinator nodes.

The number of write requests per second on the coordinator nodes, that fail because not enough replicas are available.

The number of read requests per second on the coordinator nodes, analogous to client reads. Monitoring the number of requests over a given time period reveals system read workload and usage patterns.

The min, median, max, 90th, and 99th percentiles of a client reads. The time period starts when a node receives a client read request, and ends when the node responds back to the client. Depending on consistency level and replication factor, this may include the network latency from requesting the data’s replicas.

The number of read requests on the coordinator nodes that fail due to errors returned from replicas.

The number of server read timeouts per second on the coordinator nodes.

The number of read requests per second on the coordinator nodes, that fail because not enough replicas are available.

Allocated memory, guaranteed for Java nonheap.

Maximum amount that the Java nonheap can grow.

Average amount of Java nonheap memory used.

Allocated memory guaranteed for the Java heap.

Maximum amount that the Java heap can grow.

Average amount of Java heap memory used.

Number of concurrent mark sweep garbage collections performed per second.

Number of ParNew garbage collections performed per second. ParNew collections pause all work in the JVM but should finish quickly.

Average number of milliseconds spent performing CMS garbage collections per second.

Average number of milliseconds spent performing ParNew garbage collections per second. ParNew collections pause all work in the JVM but should finish quickly.

Number of G1 old generation garbage collections performed per second.

Average number of milliseconds spent performing G1 old generation garbage collections per second.

Number of G1 young generation garbage collections performed per second.

Average number of milliseconds spent performing G1 young generation garbage collections per second.

The live disk space used by all tables on a node.

Number of bytes compacted per second.

Number of compaction tasks completed per second.

Number of sstable scans per second that could result in a compaction.

Estimated number of compactions required to achieve the desired state. This includes the pending queue to the compaction executor and additional tasks that may be created from their completion.

Aggregate of thread pools pending queues that can be used to identify where things are backing up internally. This doesn’t include pending compactions because it includes an estimate outside of the task queue or the hinted hand off queue, which can be in constant state of being on.

Aggregate of all messages that have been dropped server-side due to not having been processed before their respective timeout.

Mutation was seen after the timeout (write_request_timeout_in_ms) so was thrown away. This client might have timed out before it met the required consistency level, but might have succeeded as well. Hinted handoffs and read repairs should resolve inconsistencies but a repair can ensure it.

Mutation was seen after the timeout (write_request_timeout_in_ms) so was thrown away. This client might have timed out before it met the required consistency level, but might have succeeded as well. Hinted handoffs and read repairs should resolve inconsistencies but a repair can ensure it.

A local read request was received after the timeout (read_request_timeout_in_ms) so it was thrown away because it would have already either been completed and sent to client or sent back as a timeout error.

A local ranged read request was received after the timeout (range_request_timeout_in_ms) so it was thrown away because it would have already either been completed and sent to client or sent back as a timeout error.

The Mutation was seen after the timeout (write_request_timeout_in_ms) so was thrown away. With the read repair timeout, the node still exists in an inconsistent state.

Number of memtables queued for the flush process. A flush sorts and writes the memtables to disk.

Number of gossip messages and acknowledgments queued and waiting to be sent or received.

Number of pending tasks from internal tasks, such as nodes joining and leaving the cluster.

Repair tasks pending, such as handling the merkle tree transfer after the validation compaction.

Tasks pending to clean row caches during a cleanup compaction.

Tasks related to the last step in flushing memtables to disk as SSTables. Includes removing unnecessary commitlog files and committing Solr-based secondary indexes.

Number of pending tasks from system methods that modified the schema.

Number of pending tasks from infrequently run operations, such as taking a snapshot or processing the notification of a completed replication.

Number of pending callbacks to execute after a task on a remote node completes.

Pending task to read data from sstables and generate a merkle tree for a repair.

Pending compactions that are known. This metric could deviate from "pending compactions," which includes an estimate of tasks that these pending tasks might create after completion.

Pending tasks to calculate the ranges according to bootstrapping and leaving nodes.

Up to memtable_flush_writers concurrent tasks to flush and write the memtables to disk.

Number of gossip messages and acknowledgments actively being sent or received.

Number of active tasks from internal tasks, such as nodes joining and leaving the cluster.

Repair tasks active, such as handling the merkle tree transfer after the validation compaction.

Tasks to clean row caches during a cleanup compaction.

Tasks related to the last step in flushing memtables to disk as SSTables. Includes removing unnecessary commitlog files and committing Solr-based secondary indexes.

Number of active tasks from system methods that modified the schema.

Number of active tasks from infrequently run operations, such as taking a snapshot or processing the notification of a completed replication.

Number of callbacks to being executed after a task on a remote node is completed.

Active task to read data from sstables and generate a merkle tree for a repair.

Active compactions that are known.

Active tasks to calculate the ranges according to bootstrapping and leaving nodes.

Number of memtables flushed to disk since the nodes start.

Number of gossip messages and acknowledgments recently sent or received.

Number of recently completed tasks from internal tasks, such as nodes joining and leaving the cluster.

Repair tasks recently completed, such as handling the merkle tree transfer after the validation compaction.

Tasks to clean row caches during a cleanup compaction.

Tasks related to the last step in flushing memtables to disk as SSTables. Includes removing unnecessary commitlog files and committing Solr-based secondary indexes.

Number of completed tasks from system methods that modified the schema.

Number of completed tasks from infrequently run operations, such as taking a snapshot or processing the notification of a completed replication.

Number of completed callbacks executed after a task on a remote node is completed.

Completed tasks to read data from sstables and generate a merkle tree for a repair.

Completed compactions.

Completed tasks to calculate the ranges according to bootstrapping and leaving nodes.

The number of key cache hits per second. This will avoid possible disk seeks when finding a partition in an SSTable. This metric only applies to SSTables created by DSE versions earlier than 6.0.

The number of key cache requests per second. This metric only applies to SSTables created by DSE versions earlier than 6.0.

The percentage of key cache lookups that resulted in a hit. This metric only applies to SSTables created by DSE versions earlier than 6.0.

The number of row cache hits per second.

The number of row cache requests per second.

The percentage of row cache lookups that resulted in a hit.

The number of clients connected using the native protocol.

Number of read requests where the number of nodes queried possibly exceeds the consistency level requested in order to check for a possible digest mismatch.

Corresponds to a digest mismatch that occurred after a completed read, outside of the client read loop.

Corresponds to the number of times there was a digest mismatch within the requested consistency level and a full data read was started.

Local write requests per second. Local writes update the table’s memtable and appends to a commitlog.

The min, median, max, 90th, and 99th percentile of the response times to write data to a table’s memtable. The elapsed time from when the replica receives the request from a coordinator and returns a response.

Local read requests per second. Local reads retrieve data from a table’s memtable and any necessary SSTables on disk.

The min, median, max, 90th, and 99th percentile of the response time to read data from the memtable and sstables for a specific table. The elapsed time from when the replica receives the request from a coordinator and returns a response.

Disk space used by live SSTables. There might be obsolete SSTables not included.

Disk space used by a table by SSTables, including obsolete ones waiting to be garbage collected.

Total number of SSTables for a table.

The min, median, max, 90th, and 99th percentile of how many SSTables are accessed during a read. Includes sstables that undergo bloom-filter checks, even if no data is read from the sstable.

The min, median, max, 90th, and 99th percentile of the size (in bytes) of partitions of this table.

The min, median, max, 90th, and 99th percentile of how many cells exist in partitions for this table.

The total size of all the SSTables' bloom filters for this table.

Number of bloom filter false positives per second.

Percentage of bloom filter lookups that resulted in a false positive.

Requests per second made to a specific Solr core/index.

Average time a search query takes in a DSE cluster using DSE Search.

Errors per second that occur for a specific Solr core/index.

Timeouts per second on a specific Solr core/index.

Size of the Solr core on disk.

Stacked graph of used, cached, and free memory.

Stacked graph of used and free memory.

Total system memory currently free.

Total system memory currently used.

Total amount of memory in shared memory space.

Total system memory currently buffered.

Total system memory currently cached.

Stacked graph of committed, cached, paged, non-paged, and free memory.

Available physical memory.

Memory in use by the operating system.

Allocated pool-paged-resident memory.

Allocated pool-nonpaged memory.

Memory used by the file cache.

Stacked graph of iowait, steal, nice, system, user, and idle CPU usage.

Stacked graph of idle, user, and system CPU usage.

Stacked graph of user, privileged, and idle CPU usage.

Time the CPU devotes to user processes.

Time the CPU devotes to system processes.

Time the CPU is idle.

Time the CPU devotes to waiting for I/O to complete.

Time the CPU devotes to tasks stolen by virtual operating systems.

Time the CPU devotes to processing nice tasks.

Time the CPU devotes to processing privileged instructions.

Operating system load average. One minute value parsed from /proc/loadavg on Linux systems.

Disk space used by Cassandra at a given time.

Free space on a specific disk partition.

Disk space used by Cassandra at a given time.

Average disk throughput for read operations.

Average disk throughput for write operations.

Average disk throughput for read and write operations.

Rate of reads per second to the disk.

Rate of writes per second to the disk.

Average completion time of each request to the disk.

Average size of read requests issued to the disk.

Average size of read requests issued to the disk.

Average number of requests queued due to disk latency issues.

CPU time consumed by disk I/O.

Speed of data received from the network.

Speed of data sent across the network.

Speed of data sent across the network.

Speed of data received from the network.

Number of speculative retries for all column families.

Number of speculative retries for this table.

Rate at which new individual hints are stored on this node, to be replayed to peers.

Total off heap memory used by bloom filters from all live SSTables in a table.

Total off heap memory used by the index summary of all live SSTables in a table.

Total off heap memory used by the compression metadata of all live SSTables in a table.

Waits for current reads to complete and then frees the memory formerly used by the obsoleted memtables.

Waits for current reads to complete and then frees the memory formerly used by the obsoleted memtables.

Waits for current reads to complete and then frees the memory formerly used by the obsoleted memtables.

Off heap memory used by a table’s current memtable.

An estimate of the space used in JVM heap memory for all memtables. This includes ones that are currently being flushed and related secondary indexes.

An estimate of the space used for 'live data' (off-heap, excluding overhead) for all memtables. This includes ones that are currently being flushed and related secondary indexes.

An estimate of the space used in off-heap memory for all memtables. This includes ones that are currently being flushed and related secondary indexes.

The percentage of memory allocated for in-memory tables currently in use.

Approximate number of partitions. This may be off given duplicates in memtables and sstables are both counted and there is a very small error percentage inherited from the HyperLogLog data structure.

Deprecated. The median response times (in milliseconds) of a client write. The time period starts when a node receives a client write request, and ends when the node responds back to the client. Depending on consistency level and replication factor, this may include the network latency from writing to the replicas.

Deprecated. The median response times (in milliseconds) of a client read. The time period starts when a node receives a client read request, and ends when the node responds back to the client. Depending on consistency level and replication factor, this may include the network latency from requesting the data’s replicas.

The min, median, max, 90th, and 99th percentiles of the time from when base mutation is applied to memtable until CL.ONE is achieved on the async write to the tables materialized views. An estimate to determine the lag between base table mutations and the views consistency.

Number of view mutations sent to replicas that have been acknowledged.

Number of view mutations sent to replicas where the replicas acknowledgement hasn’t been received.

Pending tasks to send the stored hinted handoffs to a host.

Up to max_hints_delivery_threads tasks, each dispatching all hinted handoffs to a host.

Number of tasks to transfer hints to a host that have completed.

Any initialization work when a new index instance is created. This may involve costly operations such as (re)building the index.

Any initialization work when a new index instance is created. This may involve costly operations such as (re)building the index.

Any initialization work when a new index instance is created. This may involve costly operations such as (re)building the index.

The min, median, max, 90th, and 99th percentile of how many tombstones are read during a read.

Deprecated. Median response time to write data to a table’s memtable. The elapsed time from when the replica receives the request from a coordinator and returns a response.

Deprecated. Median response time to read data from the memtable and SSTables for a specific table. The elapsed time from when the replica receives the request from a coordinator and returns a response.

The min, median, max, 90th, and 99th percentiles of client reads on this table. The time period starts when a node receives a client read request, and ends when the node responds back to the client. Depending on consistency level and replication factor, this may include the network latency from requesting the data’s replicas.

The number of read requests per second for a particular table on the coordinator nodes. Monitoring the number of requests over a given time period reveals table read workload and usage patterns.

The min, median, max, 90th, and 99th percentile of how many cells were scanned during a read.

The min, median, max, 90th, and 99th percentile of how many cells were scanned during a read.

Disk space used by a table by SSTables for the tier.

Number of SSTables in a tier for a table.

Timestamp in local server time that represents an upper bound to the newest piece of data stored in the SSTable. When a new SSTable is flushed, it is set to the time of creation. When an SSTable is created from compaction, it is set to the max of all merged SSTables.

Number of hits against the adjacency cache for this graph.

Number of misses against the adjacency cache for this graph.

Number of hits against the index cache for this graph.

Number of misses against the index cache for this graph.

The min, median, max, 90th, and 99th percentile of request latencies during the period.

Number of pending tasks in the GraphScheduledThreads thread pool.

Number of active tasks in the GraphScheduledThreads thread pool.

Number of tasks completed by the GraphScheduledThreads thread pool.

Number of pending tasks in the GraphSystemThreads thread pool.

Number of active tasks in the GraphSystemThreads thread pool.

Number of tasks completed by the GraphSystemThreads thread pool.

Number of pending tasks in the GremlinWorkerThreads thread pool.

Number of active tasks in the GremlinWorkerThreads thread pool.

Number of tasks completed by the GremlinWorkerThreads thread pool.

Percentage of data (uncompressed) marked as repaired across all non-system tables on a node. Tables with a replication factor of 1 are excluded.

Percentage of data (uncompressed) marked as repired for a given table on a node. This metric is only meaningful for replication factor > 1.

Rate of coordinated reads to a node where that node is not a replica for that partition. [

Rate of coordinated reads to a node where that node did not choose itself as a replica for the read request.

The number of hints currently stored on disk, to be replayed to peers.

Rate of successful individual hints replayed to this node, from other peers.

The min, median, max, 90th, and 99th percentiles of the latency of messages between nodes. The time period starts when a node sends a message and ends when the current node receives it.

The min, median, max, 90th, and 99th percentiles of the message latency between nodes in the same or different destination datacenter. This metric measures how long it takes a message from a node in the source datacenter to reach a node in the destination datacenter. Selecting a destination node within the source datacenter yields lower latency values.

Bytes of data that were inconsistent and needed synchronization.

Bytes of data checked for consistency.

Total bytes of data transferred between all nodes during synchronization.

Number of rows and range tombstones that were inconsistent and needed synchronization.

Number of rows and range tombstones checked for consistency.

Total number of rows and range tombstones transferred between all nodes during synchronization.

Number of pages (internal groupings of data) processed.

Number of processed pages that were not in need of synchronization.

Number of processed pages that were in need of synchronization.

Number of in sync pages for which a response was gotten from only a partial number of replicas.

Number of repaired pages for which a response was gotten from only a partial number of replicas.

Number of processed pages not having enough responses to perform synchronization.

Number of processed pages for which an unknown error prevented proper synchronization completion.

Bytes of data that were inconsistent and needed synchronization.

Bytes of data checked for consistency.

Total bytes of data transferred between all nodes during synchronization.

Number of rows and range tombstones that were inconsistent and needed synchronization.

Number of rows and range tombstones checked for consistency.

Total number of rows and range tombstones transferred between all nodes during synchronization.

Number of pages (internal groupings of data) processed. [[nodesync-tbl-full-in-sync-pages]

Number of processed pages that were not in need of synchronization.

Number of processed pages that were in need of synchronization.

Number of in sync pages for which a response was gotten from only a partial number of replicas.

Number of repaired pages for which a response was gotten from only a partial number of replicas.

Number of processed pages not having enough responses to perform synchronization.

Number of processed pages for which an unknown error prevented proper synchronization completion.

Authentication Active

Authentication Completed

Execute Statement Active

Execute Statement Completed

Unknown Active

Unknown Completed

Truncate Active

Truncate Completed

Counter Acquire Lock Active

Counter Acquire Lock Completed

Write Switch For Memtable Active

Write Switch For Memtable Completed

Read Disk Async Active

Read Disk Async Completed

Timed Unknown Active

Timed Unknown Completed

Timed Timeout Active

Timed Timeout Completed

Write Defragment Active

Write Defragment Completed

Read Secondary Index Active

Read Secondary Index Completed

Total Write Defragment Blocked

Write Defragment Pending

View Build Executor Active

View Build Executor Completed

View Build Executor Pending

Background IO Stage Active

Background IO Stage Completed

Background IO Stage Pending

Repair Tasks Active

Repair Tasks Completed

Repair Tasks Pending

Auth Stage Active

Auth Stage Completed

Auth Stage Pending

Total Write Remote Blocked

Frame Decode Completed

Read Local Pending

Write Remote Completed

Read Remote Pending

Read Range Remote Pending

Read Range Local Completed

Write Local Pending

Read Range NodeSync Active

Network Backpressure Active

Read Range NodeSync Completed

Read Local Active

Read Timeout Completed

Read Internal Completed

Read Local Completed

Total Read Range Local Blocked

Read Range Internal Completed [

Hint Dispatch Active

Write Internal Active

Hint Response Completed

Write Internal Completed

Read Speculate Active

Network Backpressure Completed

Read Range Remote Completed

Read Timeout Active

Read Internal Active

Read Speculate Completed

Hint Dispatch Completed

Write Local Completed

Total Read Range Remote Blocked

Total Hint Dispatch Blocked

Total Read Local Blocked

Read Range Internal Active

Read Remote Active

Write Remote Active

Frame Decode Active

Read Range Local Active

Network Backpressure Pending

Write Local Active

Read Range Local Pending

Hint Response Active

Total Read Range NodeSync Blocked

Read Range Remote Active

Total Network Backpressure Blocked

Total Write Local Blocked

Write Remote Pending

Read Range NodeSync Pending

Hint Dispatch Pending

Total Read Remote Blocked

Read Remote Completed

Authorization Completed

Batch Replay Completed

Write Await Commitlog Segment Active

Read Switch For Response Active

NodeSync Validation Active

Read Switch For Iterator Active

Batch Remove Active

Batch Replay Active

Read Range Switch For Response Active

Write Switch For Response Active

Batch Remove Completed

Batch Store Response Completed

Write Memtable Full Active

Lwt Propose Pending

Write Await Commitlog Sync Active

NodeSync Validation Completed

Lwt Commit Completed

Read Switch For Response Completed

Lwt Prepare Active

Lwt Propose Completed

Batch Store Pending

Read Switch For Iterator Completed

Lwt Prepare Pending

Write Memtable Full Completed

Truncate Pending

Write Switch For Response Completed

Lwt Propose Active

Lwt Prepare Completed

Authorization Active

Batch Store Completed

Batch Store Active

Batch Remove Pending

Lwt Commit Active

Lwt Commit Pending

Write Await Commitlog Segment Completed

Read Range Switch For Response Completed

Batch Store Response Active

Write Await Commitlog Sync Completed

Total Batch Remove Blocked

Total Lwt Commit Blocked

Total Lwt Propose Blocked

Total Truncate Blocked

Total Lwt Prepare Blocked

Total Batch Store Blocked

Mutation of Materialized View was seen after the timeout (write_request_timeout_in_ms) so was thrown away. This client might have timed out before it met the required consistency level, but might have succeeded as well. Hinted handoffs and read repairs should resolve inconsistencies but a repair can ensure it.

Lightweight Transaction was seen after the timeout (write_request_timeout_in_ms) so was thrown away. This client might have timed out before it met the required consistency level, but might have succeeded as well. Hinted handoffs and read repairs should resolve inconsistencies but a repair can ensure it.

Hinted Handoff was seen after the timeout (write_request_timeout_in_ms) so was thrown away. Repairing the data or using NodeSync, should resolve data inconsistencies.

Truncate operation was seen after the timeout (truncate_request_timeout_in_ms) so was thrown away.

Snapshot Request was seen after the timeout (request_timeout_in_ms) so was thrown away. Snapshot should be retried.

Schema change was seen after the timeout (request_timeout_in_ms) so was thrown away. Schema agreement may not have been reached immediately, but this will eventually resolve itself.

Repair message was seen after the timeout so was thrown away.

Miscellaneous message was seen after the timeout so was thrown away.

NodeSync message was seen after the timeout so was thrown away.

Batch store message was seen after the timeout so was thrown away.

Continuous Paging Completed

Continuous Paging Active [][active-populate-virtual-table]

Populate Virtual Table Active

Populate Virtual Table Completed

Number of pending tasks in the AntiCompactionExecutor thread pool.

Number of active tasks in the AntiCompactionExecutor thread pool.

Number of tasks completed by the AntiCompactionExecutor thread pool.

Number of currently blocked tasks in the AntiCompactionExecutor thread pool.

Total number of blocked tasks in the AntiCompactionExecutor thread pool.

Number of pending tasks in the Sampler thread pool.

Number of active tasks in the Sampler thread pool.

Number of tasks completed by the Sampler thread pool.

Number of currently blocked tasks in the Sampler thread pool.

Total number of blocked tasks in the Sampler thread pool.

The min, median, max, 90th, and 99th percentile of request transversal latencies during the period.

The number of aliased traverals that have been run against the graph.

The sum of all latency during traversals of the graph.

The min, median, max, 90th, and 99th percentile of request transversal latencies during the period.

The number of aliased traversals that have been run against the graph.

The sum of all latency during traversals of graphs.

The min, median, max, 90th, and 99th percentile of request transversal latencies during the period.

The sum of all latency during unaliased traversals of graphs.

The number of unaliased traverals that have been run against the graph

The oldest data validated by Incremental NodeSync in seconds for a given time interval

Storage Attached Index disk space used.

Number of Storage Attached Indexes.

Number of Storage Attached Indexes Building.

Number of Storage Attached Indexes Queryable.

Storage Attached Index disk space used as a ratio of base table.