Configuring the available indexing threads

Improve performance on machines that have multiple CPU cores.

DSE Search provides a multi-threaded indexing implementation to improve performance on machines having multiple CPU cores. All index updates are internally dispatched to a per-CPU core indexing thread pool and executed asynchronously. This implementation allows for greater concurrency and parallelism, but as a consequence, index requests return a response before the indexing operation is actually executed. The number of available indexing threads per Solr core is by default equal to number of available CPU cores times 2. The available threads can be configured by editing the max_solr_concurrency_per_core parameter in the dse.yaml configuration file; if set to 1, DSE Search uses the legacy synchronous indexing implementation.

Also, DSE Search provides advanced, JMX-based, configurability and visibility through the IndexPool-ks.cf (where ks.cf is the name of a DSE Search Solr core) MBean under the com.datastax.bdp namespace.

About back pressure

To maximize insert throughput, DSE/Solr buffers insert requests from Cassandra so that application insert requests can be acknowledged as quickly as possible, but if too many requests accumulate in the buffer (a configurable setting), DSE/Solr pauses or blocks incoming requests until DSE/Solr catches up with the buffered requests. In extreme cases, that pause causes a timeout to the application. For more information, see "Multi-threaded indexing in DSE Search."