Statistics gathered for objects
To identify which objects (keyspace, table, or client) are currently experiencing the highest average latencies, the Performance Service maintains two latency-ordered tables, which record the mean read/write latencies and total read/write operations on a per-node, per-table basis:
-
object_read_io_snapshot
-
object_write_io_snapshot
The two tables are essentially views of the same data, but are ordered differently. Using these tables, you can identify which data objects on the node currently cause the most write and read latency to users. Because this is time-sensitive data, if a data object sees no activity for a period, no data will be recorded for them in these tables.
In addition to these two tables, the Performance Service also keeps per-object latency information with a longer retention policy in the object_io
table.
Again, this table holds mean latency and total count values for both read and write operations, but it can be queried for statistics on specific data objects (either at the keyspace or table level).
Using this table enables you to pull back statistics for all tables on a particular node, with the option of restricting results to a given keyspace or specific table.
Table activity broken down by user is retained in the object_user_read_io_snapshot
, object_user_write_io_snapshot
and object_user_io
tables.
The first two tables are ordered according to their mean latency values, making it easy for you to quickly identify which clients are currently experiencing the highest latency on specific data objects.
Having identified the hot tables on a node, you can drill down and see a breakdown of the users accessing those objects.
These tables are refreshed periodically to provide the most up to date view of activity, whereas the user_object_io
table retains data for a longer period, enabling it to be queried by node and user with the option of restricting further by keyspace or even table.