Load balancing for distributed search queries

DSE Search uses algorithms to balance the load for distributed search queries by minimizing the number of shards that are queried and reducing the amount of data that is transferred from non-local nodes. Strategies are per search index (per core) and can be changed with dsetool set_core_property. Changes are recognized with RELOAD SEARCH INDEX and do not require restarting the node. Different search indexes can have different values.

This page describes the core properties for load balancing distributed search queries and how to change them using dsetool.

shard.set.cover.finder

The shard set cover finder calculates how to set cover for a query and specify how one node is selected over others for reading the search data.

shard.set.cover.finder=STATIC: STATIC can be faster because it requires fewer token filters. For a given index, a particular coordinator accesses the same token ranges from the respective shards. Load balancing on the client side is required to achieve uniform utilization of shards by the coordinator nodes.
shard.set.cover.finder=DYNAMIC (default): With DYNAMIC, there is no fixed distribution of shard requests for a given coordinator. Any two queries can use generate shard requests accessing different sets of token ranges. This behavior means that DYNAMIC creates many unique token filters, which can be problematic. Particularly with vnodes where there is a much greater number of possible combinations. Although DYNAMIC is the default, it might not be optimal.

In your development environment, compare the load balancing performance by running separate tests with the STATIC and DYNAMIC settings.

shard.shuffling.strategy

When shard.set.cover.finder=DYNAMIC, you can change the shard shuffling strategy to one of these values:

HOST - Shards are selected based on the host that received the query.
QUERY - Shards are selected based on the query string.
HOST_QUERY - Shards are selected by host x query.
RANDOM - Different random set of shards are selected with each request (default).
SEED - Selects the same shard from one query to another.

shard.set.cover.finder.inertia

When shard.set.cover.finder=STATIC, you can change the shard cover finder inertia value. Increasing the inertia value from the default of 1 may improve performance for clusters with more than 1 vnode and more than 20 nodes. The default is appropriate for most workloads.

Change core properties

Changing core properties is an advanced operation that sets properties in the dse-search.properties resource for the search index.

These example commands show how to change core properties for the demo keyspace and the health_data table.

To change the shard set cover finder:

dsetool set_core_property demo.health_data shard.set.cover.finder=STATIC

When shard.set.cover.finder=DYNAMIC, you can change the shard shuffling strategy:
```
dsetool set_core_property demo.health_data shard.shuffling.strategy=QUERY
```
After setting a core property value, you must reload the search index (RELOAD SEARCH INDEX):
```
RELOAD SEARCH INDEX ON demo.health_data
```
You can run set_core_property for a cluster, but you must reload the search index for each datacenter. You don’t need to reindex the specified table unless you also made schema changes.
To view the state of the properties in the dse-search.properties resource:
```
dsetool list_core_properties demo.health_data
```
Result:
```
shard.set.cover.finder=STATIC
```
Log files show the loaded DSE search properties. The dsetool list_core_properties command shows only the state of the properties in the dse-search.properties resource.