IndexPool MBean

Exposes metrics around the progress of indexing tasks. Useful for controlling task submission and flush.

dse.yaml

The location of the dse.yaml file depends on the type of installation:
Package installations /etc/dse/dse.yaml
Tarball installations installation_location/resources/dse/conf/dse.yaml

The IndexPool MBean exposes metrics around the progress of indexing tasks as they move through the pipeline, and provides a mechanism to tweak the flushing, concurrency, and back-pressure behavior of a core indexing thread pool.

The index pool MBean is useful for controlling task submission and flush with these properties:
Configurable concurrency
The maximum number of concurrent workers is predefined at construction time. The actual concurrency can be dynamically configured between 1 (synchronous execution) and the given max concurrency.
Flow control via back pressure
To reduce memory consumption in case of fast producers, back pressure throttles incoming tasks. Back-pressure is applied directly as a result of the size of the global RAM buffer.

Path

The index pool MBean is identified by the following path:
com.datastax.bdp.search.keyspace_name.table_name.IndexPool
where:
  • search is the Mbean type
  • keyspace_name.table_name is the search index (core) that the metrics reference
  • IndexPool is the MBean name
For example:

IndexPool MBean attributes that you can modify

The attributes are effective only until the node is restarted. To make the change permanent, you must change the corresponding option in dse.yaml.
FlushMaxTime
The maximum time, in milliseconds, to wait before flushing asynchronous index updates, which occurs at DSE Search commit time or at database flush time. In dse.yaml (in minutes): flush_max_time_per_core.
Concurrency
The maximum number of concurrent asynchronous indexing threads.

IndexPool MBean view-only attributes

You can get the following MBean operations:
BackPressurePauseNanos
Get the average back pressure pause.
IncomingRate
Get the 1-minute rate of ingested tasks per second.
MaxConcurrency
Get the predefined max concurrency level.
OutgoingRate
Get the 1-minute rate of processed tasks per second.
ProcessedTasks
Get the total number of processed tasks for all workers.
QueueSize
Get the current size of each processing queues.
TaskProcessingTimeNanos
Get the last processing time for all workers. Could be 0 in case the clock resolution is too coarse.
Throughput
The 1-minute rate of work throughput per second.
TotalQueueSize
Get the total size of all processing queues.