Virtual node (vnode) configuration
A description of virtual nodes (vnodes) and how to use them in different types of datacenters. Also steps for disabling vnodes.
Virtual nodes simplify many tasks in DataStax Enterprise, such as eliminating the need to determine the partition range (calculate and assign tokens), rebalancing the cluster when adding or removing nodes, and replacing dead nodes. For a complete description of virtual nodes and how they work, see Virtual nodes.
- A transaction-only datacenter running OLTP.
- A single-token architecture analytics datacenter (no vnodes).
- A search datacenter with vnodes.
Guidelines for using virtual nodes
- Allocation algorithm
- Optimizes token range distribution between nodes and racks in the
datacenter based on the keyspace replication factor (allocate_tokens_for_local_replication_factor) of the
datacenter. Distributes the token ranges proportionately using the num_tokens settings. All
systems in the datacenter should have the same
num_token
settings unless the systems performance varies between systems. To distribute more of the workload to the higher performance hardware, increase the number of tokens for those systems.The allocation algorithm efficiently balances the workload using fewer tokens; when systems are added to a datacenter, the algorithm maintains the balance. Using a higher number of tokens more evenly distributes the workload, but also significantly increases token management overhead.
CAUTION:When adding multiple nodes to the cluster using the allocation algorithm, ensure that nodes are added one at a time. If nodes are added concurrently, the algorithm assigns the same tokens to different nodes.DataStax recommends using 8 vnodes (tokens). This distributes the workload between systems with a ~10% variance and has minimal impact on performance. Set the number of vnode tokens (num_tokens
) based on the workload distribution requirements of the datacenter:Allocation algorithm workload distribution variance Replication factor 4 vnode (tokens) 8 vnode (tokens) 64 vnode (tokens) 128 vnode (tokens) 2 ~17.5% ~12.5% ~3% ~1% 3 ~14% ~10% ~2% ~1% 5 ~11% ~7% ~1% ~1%
Enabling vnodes
In the cassandra.yaml file:
- Uncomment num_tokens and set the required number of tokens.
- (Recommended) To use the allocation algorithm uncomment allocate_tokens_for_local_replication_factor and set it to the target replication factor for the keyspaces in the datacenter. If the replication varies, alternate between the replication factor (RF) settings.
- Comment out the initial_token or leave unset.
To upgrade existing clusters to vnodes, see Enabling virtual nodes on an existing production cluster.
Disabling vnodes
- In the cassandra.yaml file:
- Comment out the num_tokens and allocate_tokens_for_local_replication_factor.
- Uncomment the initial_token and set it to 1 or to the value of a generated token for a multi-node cluster.
The location of the cassandra.yaml file depends on the type of installation:Installer-Services /etc/dse/cassandra/cassandra.yaml Package installations /etc/dse/cassandra/cassandra.yaml Installer-No Services install_location/resources/cassandra/conf/cassandra.yaml Tarball installations install_location/resources/cassandra/conf/cassandra.yaml