Pending task metrics for cluster operations

Pending task metrics for cluster operations can indicate a backup of cluster operational processes such as those maintaining node consistency, system schemas, fault detection, and inter-node communications.

Pending task metrics for cluster operations can indicate a backup of cluster operational processes such as those maintaining node consistency, system schemas, fault detection, and inter-node communications. Pending tasks for resource-intensive operations such as repair, bootstrap, or decommission are normal and expected while that operation is in progress, but should continue decreasing at a steady rate in a healthy cluster.

TP: Background IO Stage Pending [pending-background-io-stage]
Background IO Stage Pending
TP: Batch Remove Pending [pending-batch-remove]
Batch Remove Pending
TP: Batch Store Pending [pending-batch-store]
Batch Store Pending
TP: Cache Cleaning Pending [pending-cache-cleanup-stage]
Tasks pending to clean row caches during a cleanup compaction.
TP: Compaction Executor Pending [pending-compaction-executor]
Pending compactions that are known. This metric could deviate from "pending compactions," which includes an estimate of tasks that these pending tasks might create after completion.
TP: Flushes Pending [pending-flushes]
Number of memtables queued for the flush process. A flush sorts and writes the memtables to disk.
TP: Frame Decode Pending [pending-frame-decode]
Frame Decode Pending
TP: Gossip Tasks Pending [pending-gossip-stage]
Number of gossip messages and acknowledgments queued and waiting to be sent or received.
TP: Hint Dispatcher Pending [pending-hint-dispatcher]
Pending tasks to send the stored hinted handoffs to a host.
TP: Index Management Pending [pending-secondary-index-management]
Any initialization work when a new index instance is created. This may involve costly operations such as (re)building the index.
TP: Internal Responses Pending [pending-internal-response-stage]
Number of pending tasks from internal tasks, such as nodes joining and leaving the cluster.
TP: Manual Repair Tasks Pending [pending-anti-entropy-stage]
Repair tasks pending, such as handling the merkle tree transfer after the validation compaction.
TP: Memtable Reclaims Pending [memtable-reclaim-pending]
Waits for current reads to complete and then frees the memory formerly used by the obsoleted memtables.
TP: Migrations Pending [pending-migration-stage]
Number of pending tasks from system methods that modified the schema.
TP: Misc. Tasks Pending [pending-misc-stage]
Number of pending tasks from infrequently run operations, such as taking a snapshot or processing the notification of a completed replication.
TP: Pending Range Calculator Pending [pending-pending-range-calculator]
Pending tasks to calculate the ranges according to bootstrapping and leaving nodes.
TP: Pending Range Calculator Active [active-pending-range-calculator]
Active tasks to calculate the ranges according to bootstrapping and leaving nodes.
TP: Post Flushes Pending [pending-memtable-post-flush]
Tasks related to the last step in flushing memtables to disk as SSTables. Includes removing unnecessary commitlog files and committing Solr-based secondary indexes.
TP: Read Local Pending [pending-read-local]
Read Local Pending
TP: Read Range Remote Pending [pending-read-range-remote]
Read Range Remote Pending
TP: Read Remote Pending [pending-read-remote]
Read Remote Pending
TP: Read Repair Tasks Pending [pending-read-repair-stage]
Number of read repair operations in the queue waiting to run.
TP: Request Responses Pending [pending-request-response-stage]
Number of pending callbacks to execute after a task on a remote node completes.
TP: Validation Executor Pending [pending-validation-executor]
Pending task to read data from sstables and generate a merkle tree for a repair.
TP: View Build Executor Pending [pending-view-build-executor]
View Build Executor Pending
TP: Write Defragment Pending [pending-write-defragment]
Write Defragment Pending
TP: Write Local Pending [pending-write-local]
Write Local Pending