cassandra.yaml configuration file

The cassandra.yaml file is the main configuration file for Hyper-Converged Database (HCD).

After changing properties in the cassandra.yaml file, you must restart the node for the changes to take effect (with a few exceptions).

Syntax

For the properties in each section, the parent setting has zero spaces. Each child entry requires at least two spaces. Adhere to the YAML syntax and retain the spacing.

Default values that are not defined are shown as Default: none.

Internally defined default values are described.

HCD can define default values internally, comment them out, or create implementation dependencies on other properties in the cassandra.yaml file. Additionally, some commented-out values may not match the actual default values. DataStax recommends the commented out values as alternatives to the default values.

Quick start properties

These are the minimal properties needed for configuring a cluster:

cluster_name: The name of the cluster. This setting prevents nodes in one logical cluster from joining another so it is important to set it to a unique name other than the default. All nodes in a cluster must have the same value.

Default: 'Test Cluster'

rpc_address: The address that client applications connect to. You typically set this to a node’s public IP address that is routable from the clients.

If not changed from the default localhost, only applications deployed on the server will be able to connect to the node.

Default: localhost

storage_port

The port for all internode communication.

storage_port replaces the deprecated ssl_storage_port because storage_port handles both encrypted and unencrypted internode communication.

For security, do not expose this port to the internet. Apply firewall rules. For more information, see the following:

Default: 7000

listen_address

The IP address or hostname that the database binds to, exclusively for private communication between nodes in the cluster. This is typically set to a node’s private IP that is routable from other nodes.

If not changed from the default localhost, the node will not be able to communicate with other nodes in the cluster.

Never set listen_address to 0.0.0.0.

Set listen_address or listen_interface but not both.

The recommended usage of listen_address or listen_interface depend on your deployment architecture:

Architecture Option 1 Option 2

Architecture	Option 1	Option 2
Single-node installation	Use the default setting of `localhost`. If the node is properly configured with the host name, name resolution, and other necessary information, then the database uses `InetAddress.getLocalHost()` to get the local address from the system automatically.	Comment out `listen_address`. If the node is properly configured with the host name, name resolution, and other necessary information, then the database uses `InetAddress.getLocalHost()` to get the local address from the system automatically.
A node in a multi-node installation	Set `listen_address` to the node’s IP address or hostname.	Set `listen_interface`.
A node in a multi-network or multi-datacenter installation, deployed on EC2, and supports automatic switching between public and private interfaces:	Set `listen_address` to the node’s IP address or hostname.	Set `listen_interface`.
A node with two physical network interfaces in a multi-datacenter installation or A cluster deployed across multiple EC2 regions using the Ec2MultiRegionSnitch	Set `listen_address` to the node’s private IP or hostname for communication within the local datacenter. Set `broadcast_address` to the second IP or hostname for communication between datacenters. Set `listen_on_broadcast_address` to `true`. If the node is a seed node, add the node’s public IP address or hostname to the `seeds` list in `cassandra.yaml`.	Set `listen_interface` for communication within the local datacenter. Set `broadcast_address` to the second IP or hostname for communication between datacenters. Set `listen_on_broadcast_address` to `true`. If the node is a seed node, add the node’s public IP address or hostname to the `seeds` list in `cassandra.yaml`.

Single-node installation

Use the default setting of localhost.

If the node is properly configured with the host name, name resolution, and other necessary information, then the database uses InetAddress.getLocalHost() to get the local address from the system automatically.

Comment out listen_address.

A node in a multi-node installation

Set listen_address to the node’s IP address or hostname.

Set listen_interface.

A node in a multi-network or multi-datacenter installation, deployed on EC2, and supports automatic switching between public and private interfaces:

Set listen_address to the node’s IP address or hostname.

Set listen_interface.

A node with two physical network interfaces in a multi-datacenter installation

A cluster deployed across multiple EC2 regions using the Ec2MultiRegionSnitch

Set listen_address to the node’s private IP or hostname for communication within the local datacenter.
Set broadcast_address to the second IP or hostname for communication between datacenters.
Set listen_on_broadcast_address to true.
If the node is a seed node, add the node’s public IP address or hostname to the seeds list in cassandra.yaml.

Set listen_interface for communication within the local datacenter.
Set broadcast_address to the second IP or hostname for communication between datacenters.
Set listen_on_broadcast_address to true.
If the node is a seed node, add the node’s public IP address or hostname to the seeds list in cassandra.yaml.

Make sure the storage_port is open on the public IP firewall.

Default: localhost

listen_interface

The interface that the database binds to for connecting to other nodes. Interfaces must correspond to a single address. HCD does not support IP aliasing.

Never set listen_address to 0.0.0.0.

Set listen_address or listen_interface but not both.

For use cases, see listen_address.

listen_interface_prefer_ipv6

Use IPv4 or IPv6 when an interface is specified by name.

false: Use first IPv4 address.
true: Use first IPv6 address.

When you use only a single address, HCD selects that address without regard to this setting.

+ Default: false

Default directories

If you have changed any of the default directories during installation, set these properties to the new locations. Make sure you have root access.

data_file_directories: The directory where table data is stored on disk. The database distributes data evenly across the location, subject to the granularity of the configured compaction strategy.

For production, DataStax recommends RAID-0 and SSDs. For more information, see Capacity planning and hardware selection for HCD deployments.

Default: /var/lib/cassandra/data

commitlog_directory

The directory where HCD stores the commit log.

For optimal write performance, place the commit log on a separate disk partition, or ideally on a separate physical device, from the data directories. Because the commit log only appends data, a hard disk drive (HDD) works as long as it keeps up with the writes.

Default: $CASSANDRA_HOME/data/commitlog

The commitlog_directory and the cdc_raw_directory must reside on the same partition. Keep these directories in separate sub-folders that are not nested.

cdc_raw_directory

The directory where HCD stores change data capture (CDC) commit log segments on flush. DataStax recommends using a physical device that is separate from the data directories. See Change Data Capture (CDC) logging.

Default: $CASSANDRA_HOME/data/cdc_raw

The cdc_raw_directory and the commitlog_directory must reside on the same partition. Keep these directories in separate sub-folders that are not nested.

hints_directory: The directory where HCD stores hints (missed writes).

Default: $CASSANDRA_HOME/data/hints

metadata_directory: The directory that holds cluster metadata including information about the local node and its peers.

Default: $CASSANDRA_HOME/data/metadata

saved_caches_directory: The directory location where HCD stores saved counter and row cache keys if enabled.

Default: $CASSANDRA_HOME/data/saved_caches
local_system_data_file_directory: Directory where HCD stores the data of the local system keyspaces. This setting allows storing system keyspace data in a different directory to provide redundancy.

Default: Not set (uses the first directory in data_file_directories)

Data directory configuration

Distributing data across multiple disks, also known as Just a Bunch Of Disks (JBOD) configuration, can improve throughput and efficiency of disk I/O. HCD lets you specify multiple directories for storing your data in this distributed manner.

DataStax recommends using striped LVM instead of JBOD for disk management. For more information, see RAID on data disks.

To configure a single JBOD data directory in the cassandra.yaml file:

data_file_directories:
     - /var/lib/cassandra/data

For multiple JBOD data directories:

data_file_directories:
     - /disk1/datadir
     - /disk2/datadir
     - /disk3/datadir

Commonly used properties

These are the properties most frequently used when configuring HCD.

Before starting a node for the first time, DataStax recommends that you carefully evaluate your requirements.

Common initialization properties

Be sure to set the properties in the Quick start section as well.

commit_failure_policy

Determines how HCD handles commit log disk failures.

die: Shut down the node and kill the JVM, so the node can be replaced.
stop: Shut down the node, leaving the node effectively dead, available for inspection using JMX.
stop_commit: Shut down the commit log, letting writes collect but continuing to service reads.
ignore: Ignore fatal errors and let the batches fail.

Default: stop (recommended)

disk_optimization_strategy

Reading from spinning disks is slow so HCD buffers them with an extra page of 4KB just in case. This is unnecessary for SSDs so HCD buffers only what is required.

ssd: Data directory backed by solid state disks
spinning: Data directory backed by spinning disks

Default: ssd

disk_failure_policy

Determines how HCD handles disk failures.

die: Shut down gossip and client transports, and kill the JVM for any file system errors or single SSTable errors, so the node can be replaced.
stop_paranoid: Shut down the node, even for single SSTable errors.
stop: Shut down the node leaving the node effectively dead, but the JVM is still available for inspection using JMX.
best_effort: Stop using the failed disk and respond to requests based on the remaining available SSTables. This setting allows obsolete data at consistency level of ONE.
ignore: Ignore fatal errors and let the requests fail; all file system errors are logged but otherwise ignored.

Recommended policies are stop and best_effort. Default: stop

phi_convict_threshold: Phi value that must be reached for a host to be marked down by the failure detector.

Default: 8

endpoint_snitch

Configure this property to set the snitch. The most common snitches are:

Default: SimpleSnitch

Uses replication strategy order for proximity. This snitch does not recognize racks or datacenters, and considers all nodes as belonging to one ring (single DC) making it incompatible with multi-DC deployments and unsuitable for production environments.

This snitch is appropriate for development environments only.
GossipingPropertyFileSnitch (GPFS)

Uses rack and datacenter information for the local node defined in the cassandra-rackdc.properties file and propagates this information to other nodes via gossip.

This snitch is recommended for production environments and is almost always the correct choice.

PropertyFileSnitch (PFS)

Determines node proximity using the rack and datacenter location defined in the cassandra-topology.properties file. GPFS supersedes this snitch. Only use PFS for backwards compatibility.

For other snitches such as Ec2Snitch and GoogleCloudSnitch, see About snitches.

All nodes in a cluster must use the same snitch.

HCD determines replica placement (which defines where copies of data are stored) using the information provided by the snitch. Changing the snitch has implications for where the data is located so requires additional steps and should only be performed by experienced operators.

seed_provider

The gossip seed provider and corresponding addresses of nodes that are designated as contact points in the cluster. A joining node contacts the nodes in the seeds list and establishes a connection to the first available node to discover the members of the cluster and topology.

class_name: The class that handles the seed logic. The default is appropriate for most clusters. For specific edge cases, you can substitute a custom seed provider class.

Default: org.apache.cassandra.locator.SimpleSeedProvider

seeds: A comma delimited list of addresses and their corresponding storage_port. A new node joining the cluster uses the list to bootstrap the gossip process. If the cluster has multiple nodes, the default value must be changed to the IP address and gossip port of one of the nodes.

Default: "127.0.0.1:7000"

Making every node a seed node is not recommended because of increased maintenance and reduced gossip performance. Gossip optimization is not critical, but it is recommended to use a small seed list of approximately three nodes per datacenter.

Advanced initialization properties

allocate_tokens_for_keyspace (not recommended)

The token allocation algorithm for vnodes distributes token ranges across nodes in a datacenter based on the number of nodes and the value of num_tokens. With allocate_tokens_for_keyspace, token distribution also considers a specified replication factor, resulting in a more balanced distribution compared to legacy random allocation (without vnodes).

Requires the following:

num_tokens must be set.
partitioner must be set to the Murmur3Partitioner class.
initial_token must be commented out or not set.
allocate_tokens_for_local_replication_factor must be commented out or not set.

Set allocate_tokens_for_keyspace to the name of an existing keyspace with defined replication properties. The keyspace’s replication factor, such as 3, is used for token allocation.

allocate_tokens_for_local_replication_factor is recommended over allocate_tokens_for_keyspace because it doesn’t require a preexisting keyspace.

Default: Not set (fallback to allocate_tokens_for_local_replication_factor or, if neither allocate_tokens_* parameters are set, use legacy random allocation)

allocate_tokens_for_local_replication_factor (recommended)

The token allocation algorithm for vnodes distributes token ranges across nodes in a datacenter based on the number of nodes and the value of num_tokens. With allocate_tokens_for_local_replication_factor, token distribution also considers a specified replication factor, resulting in a more balanced distribution compared to legacy random allocation (without vnodes).

Requires the following:

num_tokens must be set.
partitioner must be set to the Murmur3Partitioner class.
initial_token must be commented out or not set.
allocate_tokens_for_keyspace must be commented out or not set.

Set allocate_tokens_for_local_replication_factor to the keyspace replication factor in the node’s local datacenter, such as 3.

allocate_tokens_for_local_replication_factor is recommended over allocate_tokens_for_keyspace because it doesn’t require a preexisting keyspace.

Default: Not set (fallback to allocate_tokens_for_keyspace)

auto_bootstrap: When joining a cluster for the first time, this property determines whether the node will request replicas to stream data. This is the default behavior. If the node is defined as a seed node, it immediately joins the cluster without data.

Non-seed nodes will bootstrap automatically by default. Set to false when adding nodes in a new datacenter where bootstrap is manually triggered by an operator with the nodetool rebuild command.

Default: true

broadcast_address: Set to the node’s public IP address in environments where nodes are only able to communicate across networks using their public IP addresses such as multi-region Amazon EC2 deployments. Otherwise, the node will broadcast on the same address as listen_address.

Set a separate listen_address and broadcast_address on a node with multiple network interfaces or where nodes are not able to communicate over private IP addresses. Not required in environments that support automatic switching between private and public communication.

Default: uses value of listen_address

initial_token

Manually assign tokens for token range allocation to nodes:

Single-token architecture: Specify one token value per node.
Virtual nodes (vnodes): Specify multiple tokens as a comma-separated list. However, num_tokens is preferred over initial_token for simplified, automated token allocation.

Default: Not set (fallback to num_tokens)

listen_on_broadcast_address: Set to true on nodes with multiple interfaces to enable communication on both listen_address and broadcast_address.

Default: false

num_tokens

When using vnodes, defines the number of token ranges to assign to the node:

1: Monolithic and inflexible when scaling.
2 to 8: Compared to 16 tokens, these values result in greater node availability (faster internode communication) but more variance in data size per node. 4 or 8 tokens can be more performant for large clusters.
16 (default): Provides high elasticity for scaling and distributes data without significant reduction in availability for all but the largest clusters.
More than 16 tokens: Rarely necessary but can be used.

If num_tokens is set:

DataStax recommends that you also set allocate_tokens_for_local_replication_factor.
initial_token must be commented out or not set.
partitioner must be set to the Murmur3Partitioner class.

partitioner: The partitioner determines how data is distributed across the nodes in the cluster.

The default Murmur3Partitioner is the correct and only choice for new clusters. The legacy partitioners are provided for backward-compatibility with existing clusters upgraded from earlier versions of Cassandra since the partitioner can never be changed on a running cluster.

Default: org.apache.cassandra.dht.Murmur3Partitioner

Common compaction settings

compaction_throughput_mb_per_sec

The rate in megabytes/second at which HCD compacts SSTable candidates. The faster the database inserts data, the faster HCD must compact in order to keep the number of SSTables down.

Set to 16 to 32 times the write throughput in MB/second. Otherwise, set to 0 to disable compaction throttling. A high setting means that HCD uses more disk I/O for compaction, leaving less I/O bandwidth for reads.

Default: 64

See Configure compaction.

Memtable settings

When a node receives a write request, HCD stores the data in a memory structure called a memtable and appends it to the commit log on disk for durability (see How data is written). HCD can allocate memtable segments either on- or off-heap.

memtable_allocation_type

Determines how HCD allocates memory to the memtable.

heap_buffers: HCD allocates memtables on JVM heap. Suitable for general workloads where heap memory is sufficient.
offheap_buffers: Uses Java NIO direct buffers to store cell names and values off-heap. This allocation type reduces heap utilization significantly, leading to reduced GC pressure.
offheap_objects: Allocates memtables completely off-heap, directly in native memory. This allocation type is recommended particularly for clusters that handle large datasets. Writes are around 5% faster mostly due to memtables flushing less often.
unslabbed_heap_buffers: Allocates memtables on JVM heap without using a slab allocator. This isn’t recommended because it can lead to increased heap fragmentation.

Default: offheap_objects

memtable_heap_space_in_mb: The maximum amount of memory to allocate for memtables on JVM heap. When the threshold is reached, writes are blocked until a flush completes.

Default: ¼ of heap

memtable_offheap_space_in_mb: The maximum amount of memory to allocate for memtables from native memory. When the threshold is reached, writes are blocked until a flush completes.

Default: ¼ of heap

memtable_cleanup_threshold: The threshold that triggers a flush based on the ratio of memtable size to the maximum memory size permitted for memtables.

Deprecated and commented out by default because the default calculation is the only reasonable choice. Only uncomment for backwards compatibility when upgrading. For more information, see the comments on this parameter in your installation’s default cassandra.yaml file.

Default: 1 / (memtable_flush_writers + 1)

memtable_flush_writers: The total number of memtables that can be flushed concurrently as well as the number of flush writer threads per disk. Factors into the memtable flush threshold calculation.

Commented out and uses the default value of 2 for nodes with a single data directory. For nodes with multiple data directories, the default behavior is 1 memtable flushed at a time (sequential individual memtable flushes).

Typically, there is no need to tune this parameter, and increasing this value can degrade performance. To determine if memtable flushes are falling behind, check if the MemtablePool.BlockedOnAllocation metric is greater than 0. If this metric is too large, then you might need to adjust this parameter.

For more information, see the comments on this parameter in your installation’s default cassandra.yaml file.

Common automatic backup settings

Backups and snapshots are not automatically cleared, which can cause unbounded increases in disk usage. By default, when the disk is full, HCD automatically shuts down because it can no longer write files to disk.

DataStax recommends setting up a process to clear incremental backups each time a new snapshot is created.

auto_snapshot: Whether a snapshot is taken before DROP KEYSPACE, DROP TABLE, or TRUNCATE TABLE is executed.

DataStax strongly recommends keeping this enabled as a precaution in case a DROP or TRUNCATE command is executed accidentally against the wrong keyspace or table.

Default: true

incremental_backups: Whether HCD creates hard links to each SSTable that has been flushed or streamed in the backups/ subdirectory of the keyspace data.

Default: false

snapshot_before_compaction: Whether a snapshot is taken before each compaction task. The snapshot may be used as a rollback position in an upgrade. Usage is limited since the general recommendation is to take backups before performing an upgrade.

Use with extreme caution as disk usage can grow exponentially.

Default: false

Performance tuning

Tuning performance and system resource utilization, including commit log, compaction, memory, disk I/O, CPU, reads, and writes.

Commit log settings

commitlog_sync

Defines the mode by which the commit log is synchronized to disk. When the data is considered fully persisted to storage, the data will survive a system crash or power outage. The sync mode also determines when HCD sends a successful write acknowledgement to the coordinator.

batch: Each write request triggers a call to sync immediately. The acknowledgement is blocked until the after the commit log has been flushed to disk. Prioritizes durability over performance.
group: Similar to batch mode but waits up to commitlog_sync_group_window_in_ms between flushes so more writes are persisted together. HCD also blocks the acknowledgement until after the commit log has been flushed to disk. Recommended over batch mode.
periodic: HCD synchronizes the commit log every commitlog_sync_period_in_ms but the write is acknowledged immediately. Prioritizes performance over durability.

Default: periodic

commitlog_sync_period_in_ms

Time interval between commit log syncs to disk. Only set with periodic sync mode, otherwise an exception will be logged.

Default: 10000 (10 seconds)

commitlog_sync_group_window_in_ms

The minimum interval between disk syncs. Only set with group sync mode, otherwise an exception will be logged.

Default: 1000 (1 second)

commitlog_sync_batch_window_in_ms

Deprecated. The maximum delay between disk syncs. No longer used.

periodic_commitlog_sync_lag_block_in_ms

When in periodic commitlog mode, the number of milliseconds to block writes while waiting for a slow disk flush to complete.

Default: Not set (no blocking)

commitlog_segment_size_in_mb: The size of individual commit log file segments. A small size means more frequent flushes leading to small SSTables which put pressure on compaction.

If you use the commit log archives for point-in-time recovery, it is reasonable to reduce the size to 16 or 8MB for finer granularity. For more information, see max_mutation_size_in_kb.

Default: 32

max_mutation_size_in_kb: The maximum allowed size of a mutation (the payload size of a write request) which defaults to half of the commit log segment (commitlog_segment_size_in_mb). If explicitly set, you must set commitlog_segment_size_in_mb to at least twice the value of max_mutation_size_in_kb.

Before increasing the commit log segment size of the commit log segments, investigate why the mutations are larger than expected. Look for underlying issues with access patterns and data model, because increasing the commit log segment size is a limited fix.

Default: ½ of commitlog_segment_size_in_mb

commitlog_total_space_in_mb

The maximum disk space for commit logs on disk.

If the limit is reached, the oldest commit log segments are flushed to reclaim disk space. A small size means more frequent flushes on less-active tables leading to small SSTables which put pressure on compaction.

Default: 8192 or ¼ of /commitlog disk, whichever is smaller

See Configure memtable thresholds.

commitlog_compression

By default, the commit log is not compressed. To enable compression, specify the compression library to use.

The supported libraries are:

DeflateCompressor: Not recommended. Legacy option that is the slowest compared to newer algorithms.
LZ4Compressor: Fastest algorithm but offers less compression ratios. Choose when speed is preferred over space savings.
SnappyCompressor: Not as fast as LZ4 but provides better compression.
ZstdCompressor: Provides the best compression ratio but slower than other algorithms.
```
commitlog_compression:
  - class_name: LZ4Compressor
```

flush_compression

Compression to apply to SSTables as they flush for compressed tables. High-ratio compressors can potentially block flushes for too long, so the default is to flush with a known fast compressor in those cases.

Available options:

fast (default): Flush with a fast compressor. If the table is already using a fast compressor, that compressor is used.
none: Flush without compressing blocks but while still doing checksums.
table: Always flush with the same compressor that the table uses.

hints_compression

The compressor for hint files. When not set, the database does not compress hints files.

To enable compression, specify the compressor class name. The supported compressors are:

LZ4Compressor: Fastest algorithm, but offers lower compression ratios.
SnappyCompressor: Not as fast as LZ4, but provides higher compression ratios.
DeflateCompressor: Not recommended. Legacy option that is the slowest compared to newer algorithms.

Default: Compression disabled

hints_compression:
  - class_name: LZ4Compressor
    parameters:
     - lz4_compressor_type: fast

hinted_handoff_enabled

Whether hinted handoff is enabled globally. When false, no hints are generated for any unavailable replica.

Default: false

hinted_handoff_disabled_datacenters

A list of datacenters that will not perform hinted handoff when hinted_handoff_enabled is true.

hinted_handoff_disabled_datacenters:
   - DC1
   - DC2

max_hint_window_in_ms: The maximum amount of time in milliseconds a dead host will have hints generated. After it has been dead this long, new hints for it will not be created until it has been seen alive and gone down again.

Default: 10800000 (3 hours)
hinted_handoff_throttle_in_kb: Maximum throttle in KB per second, per delivery thread. This is reduced proportionally to the number of nodes in the cluster.

Default: 1024
max_hints_delivery_threads: Number of threads with which to deliver hints. Consider increasing this number for multi-datacenter deployments, since cross-datacenter handoff tends to be slower.

Default: 2
hints_flush_period_in_ms: How often hints should be flushed from the internal buffers to disk in milliseconds. Will not trigger fsync.

Default: 10000 (10 seconds)
max_hints_file_size_in_mb: Maximum size for a single hints file, in megabytes.

Default: 128

Change Data Capture (CDC) settings

Compaction settings

See also compaction_throughput_mb_per_sec in the common compaction settings section and Configure compaction.

concurrent_compactors

The number of compaction threads allowed to run simultaneously. Simultaneous compactions help preserve read performance in a mixed read-write workload by limiting the number of small SSTables that accumulate during a single long-running compaction.

Generally, the calculated default value is appropriate and does not need adjusting. DataStax recommends contacting IBM Support before changing this value. If your data directories are backed by solid-state drives (SSDs), increase this value to the number of cores.

If compaction runs too slowly or too fast, adjust the compaction_throughput_mb_per_sec option.

Increasing concurrent compactors leads to more use of available disk space for compaction, because concurrent compactions happen in parallel, especially for STCS. Ensure that adequate disk space is available before increasing this configuration.

Default: fewer of number of data disks or CPU cores, with a minimum of 2 and a maximum of 8

concurrent_validations

The number of repair validation threads allowed to run simultaneously.

Defaults to the value of concurrent_compactors if not configured or set to ⇐ 0. Requires system property -Dcassandra.allow_unlimited_concurrent_validations=true to set validation threads to a value higher than concurrent compactors.

Default: the value of concurrent_compactors

concurrent_reads

The number of concurrent read threads. For workloads with more data than fits in memory, set this value to 16 times the number of drives to allow operations to enqueue low enough in the stack that the OS and drives can reorder them.

Default: 32

concurrent_writes

The number of concurrent write threads. Because writes are almost never I/O bound, the ideal number depends on the number of cores in your system (8 times the number of cores is a good rule of thumb.)

Default: 32

concurrent_counter_writes

The number of concurrent counter write threads. Counter writes perform a read-before-write step to fetch current values prior to incrementing and persisting them back to storage, making them I/O-bound similar to regular reads.

DataStax recommends setting this to the same value as concurrent_reads. You might set this to a lower value if your workload does not use counter columns, or if you need to throttle counter mutations to prevent them from competing with standard reads for disk I/O.

Default: 32

concurrent_materialized_view_writes

The number of concurrent materialized view write threads. Because materialized view writes include a read step, this should be limited to the lesser of concurrent_reads or concurrent_writes.

Default: 32

concurrent_materialized_view_builders

The number of view builder tasks allowed to run simultaneously if materialized views are enabled. This is experimental.

When a view is created, the node ranges are split into [num_processors x 4] builder tasks. Set this property to 2 or higher to build views faster.

Default: 1

sstable_preemptive_open_interval_in_mb

The size of the SSTable candidates to trigger preemptive opening of compaction output.

The compaction process opens SSTables before HCD completely writes them and uses them in place of the prior SSTables for any range previously written. Preemptive opening of SSTables helps to smoothly transfer reads between the SSTables by reducing cache churn and keeps hot rows hot.

A low value has a negative performance impact and will eventually cause heap pressure and GC activity. The optimal value depends on hardware and workload.

Default: 50

Cache and index settings

column_index_size_in_kb: Granularity of the index of rows within a partition. For huge rows, decrease this setting to improve seek time.

Default: 64
prepared_statements_cache_size_mb: Maximum size of the native protocol prepared statement cache in MB. Valid values are "auto" (omitting the value) or a value greater than 0.

Default: 1/256th of heap or 10MB, whichever is greater
key_cache_size_in_mb: Maximum size of the key cache in memory in MB. Each key cache hit saves 1 seek.

Set to 0 to disable key cache.

Default: Automatically calculated as 5% of heap or 100MB, whichever is smaller.
key_cache_save_period: Duration in seconds after which HCD should save the key cache to saved_caches_directory. Saved caches greatly improve cold-start speeds with relatively low I/O cost.

Default: 14400 (4 hours)
key_cache_keys_to_save: Number of keys from the key cache to save.

Default: 0 (disabled by default, meaning all keys are saved)
networking_cache_size_in_mb: Maximum memory (in MB) to use for internode and client-server networking buffers. This buffer pool is allocated off-heap and is in addition to configured JVM heap memory. Memory is only allocated as needed.

Default: 1/16 of heap or 128MB, whichever is smaller
file_cache_enabled: Whether the SSTable chunk cache is enabled. The chunk cache stores recently accessed sections of the SSTable in memory as uncompressed buffers.

Default: false

file_cache_size_in_mb

Maximum memory (in MB) to use for SSTable chunk cache and buffer pooling. 32 MB is reserved for pooling buffers; the rest is used for the chunk cache that holds uncompressed SSTable chunks. This buffer pool is allocated off-heap and is in addition to configured JVM heap memory. Memory is only allocated as needed.

Default: 1/4 of heap or 512MB, whichever is smaller

buffer_pool_use_heap_if_exhausted

Whether to allocate on or off heap when the SSTable buffer pool is exhausted (that is, when it has exceeded file_cache_size_in_mb).

Default: true

cache_load_timeout_seconds

Number of seconds the server will wait for each cache (row, key, counter, etc.) to load while starting the process. Setting this to a negative value disables all cache loading on startup while still having the cache during runtime.

Default: 30

repair_session_space_in_mb

Limit memory usage in MB for Merkle tree calculations during repairs. The main tradeoff is that smaller trees have less resolution, which can lead to over-streaming data.

Default: 1/16 of available heap

index_summary_capacity_in_mb

A fixed memory pool size in MB for SSTable index summaries. If left empty, this defaults to 5% of the heap size. If the memory usage of all index summaries exceeds this limit, SSTables with low read rates shrink their index summaries.

Default: 5% of heap size

index_summary_resize_interval_in_minutes

How frequently index summaries should be resampled in minutes to redistribute memory from the fixed-size pool to SSTables proportional to their recent read rates. Setting to -1 disables this process.

Default: 60

column_index_cache_size_in_kb

Only applies to big format SSTables, which have a partition key cache.

Set the threshold for the total size of all index entries for a partition that the database stores in the partition key cache. If the total size of all index entries for a partition exceeds this amount, the database stops putting entries for this partition into the partition key cache.

Default: 2

row_cache_class_name

The classname of the row cache provider to use if row_cache_size_in_mb and row_cache_save_period aren’t 0:

org.apache.cassandra.cache.OHCProvider (default): Fully off-heap.
org.apache.cassandra.cache.SerializingCacheProvider (legacy): Partially off-heap. Provided for backwards compatibility.

Use only row cache provider implementations bundled with HCD.

row_cache_size_in_mb

Maximum size of the row cache in memory. The row cache can improve read latency, but it requires a significant amount of memory because it contains the entire row. Use the row cache only for frequently read rows or static rows. If the cache is too small, it might be too cold at startup and perform suboptimally until it has warmed up and recached the most frequently read rows.

0 (default): Disable the row cache.
Positive number: Set the maximum size in MB of the row cache in memory.

row_cache_save_period

The time in seconds after which row cache keys are saved to saved_caches_directory. Only applies if row_cache_size_in_mb is greater than 0.

0 (default): Disables saving the row cache. Saving the row cache isn’t required to use the row cache, and it isn’t always recommended. Although the row cache will be cold at startup, you can warm it up by running preemptive reads on frequently accessed rows before accepting application traffic.
Positive number: The time in seconds after which a row cache key is saved.

row_cache_keys_to_save

The number of keys from the row cache to save if row_cache_save_period is greater than 0.

The value 0 has a different meaning for row_cache_keys_to_save compared to other row_cache_* settings.

0: All keys are saved.
Positive number: Saves the specified number of keys from the row cache.

Default: 100

counter_cache_size_in_mb

The size of the counter cache. It doesn’t require as much memory as other caches because only the local (clock, count) tuple of a counter cell is kept in memory, not the whole counter.

Not set or empty (default): Automatically calculated as 2.5 percent of heap or 50 MB, whichever is smaller.
Positive number: The size in MB of the counter cache in memory. If the cache is too small, it might be too cold at startup and perform suboptimally until it has warmed up and recached the most frequently updated counters.
0: Disable the counter cache. The counter cache isn’t recommended for workloads with counter deletes that rely on a low gc_grace_seconds.

counter_cache_save_period

The time in seconds after which the database saves counter cache keys to saved_caches_directory. This keeps the cache warm through restarts.

0: Disables saving the counter cache. This makes the cache cold at startup and the first writes will run slower because they must perform a full read-before-write and populate the cache.
Positive number: The time in seconds after which a counter cache key is saved.

Default: 7200 (2 hours)

counter_cache_keys_to_save

The number of keys from the counter cache to save if counter_cache_save_period is greater than 0.

The value 0 has a different meaning for counter_cache_keys_to_save compared to other counter_cache_* settings.

0 (default): All keys are saved.
Positive number: Saves the specified number of keys from the counter cache.

Streaming settings

These settings apply to operations that perform file streaming, including repairs, bootstraps, and decommissions. These operations are mostly sequential I/O, which can saturate a node’s network bandwidth and degrade client (application) performance. Therefore, it is important to throttle streaming throughput.

inter_dc_stream_throughput_outbound_megabits_per_sec: Maximum network bandwidth for streaming file transfers between datacenters. Set to a value less or equal to stream_throughput_outbound_megabits_per_sec.

Default: 200 Mbps (25 MB/s)

stream_entire_sstables: Whether Zero Copy Streaming is enabled, where eligible SSTables are streamed in their entirety between nodes instead of individual partitions, transferring data at a significantly faster rate.

This feature is bound to the streaming throughput limits and disabled when internode encryption is enabled.

Default: true

stream_throughput_outbound_megabits_per_sec: Maximum network bandwidth permitted for all outbound streaming file transfers on a node.

Default: 200 Mbps (25 MB/s)

streaming_keep_alive_period_in_secs: Interval to send keep-alive messages to prevent reset connections during streaming. The streaming session fails when a keep-alive message is not received for two keep-alive cycles equivalent to 10 minutes by default (2 x 300 seconds).

Default: 300
streaming_connections_per_host: Limit number of connections per host for streaming. Increase this when joins are CPU-bound rather than network-bound (for example, a few nodes with large files).

Default: 1

Advanced properties

Less commonly-used settings normally reserved for experienced operators.

ssl_storage_port

SSL port for legacy encrypted communication. Deprecated as of Cassandra 4.0 because a single storage_port can be used for both secure and insecure connections. For security reasons, do not expose this port to the internet.

Default: 7001

listen_interface

The network interface that HCD binds to for connecting to other HCD nodes. Set listen_address or listen_interface, not both. Interfaces must correspond to a single address; IP aliasing is not supported.

listen_interface_prefer_ipv6

If you use listen_interface to specify the interface by name and the interface has both an IPv4 and IPv6 address, controls whether IPv4 (false) or IPv6 (true) is chosen.

Default: false

broadcast_rpc_address

RPC address to broadcast to drivers and other HCD nodes. Cannot be set to 0.0.0.0. If left blank, this is set to the value of rpc_address by default.

rpc_interface

The network interface that HCD listens on for client connections. Use this instead of rpc_address when you want to specify the interface by name rather than by IP address. Set rpc_address or rpc_interface, not both. Interfaces must correspond to a single address; IP aliasing is not supported.

rpc_interface_prefer_ipv6

If you use rpc_interface to specify the interface by name and the interface has both an IPv4 and IPv6 address, controls whether IPv4 (false) or IPv6 (true) is chosen.

Default: false

native_transport_port

Port for the CQL native transport to listen for clients on. For security reasons, do not expose this port to the internet.

Default: 9042

native_transport_port_ssl

Enabling native transport encryption in client_encryption_options allows you to either use encryption for the standard port or use a dedicated additional port along with the unencrypted standard native_transport_port. Setting native_transport_port_ssl to a different value from native_transport_port uses encryption for native_transport_port_ssl while keeping native_transport_port unencrypted.

Default: 9142

native_transport_max_threads

The maximum threads for handling native transport requests. Idle threads are stopped after 30 seconds.

Default: 128

native_transport_max_frame_size_in_mb

The maximum size of allowed native transport frame in MB. Frames (requests) larger than this will be rejected as invalid. Must be positive and less than 2048.

Default: 256

native_transport_max_concurrent_connections

The maximum number of concurrent client connections. The default of -1 means unlimited.

Default: -1

native_transport_max_concurrent_connections_per_ip

The maximum number of concurrent client connections per source IP address. The default of -1 means unlimited.

Default: -1

native_transport_allow_older_protocols

Whether HCD honors older, yet currently supported, protocol versions.

Default: true

native_transport_idle_timeout_in_ms

When idle client connections are closed. Idle connections are ones that have had neither reads nor writes for this time period in milliseconds.

Default: 0 (disabled)

native_transport_flush_in_batches_legacy

Whether to use native transport TCP message coalescing. If on upgrade you find throughput decreasing with an older kernel or very few client connections, evaluate this option.

Default: false

internode_socket_send_buffer_size_in_bytes

Socket buffer size for internode communication send buffer. When not set, buffer size is defined by the OS default (net.ipv4.tcp_wmem).

internode_socket_receive_buffer_size_in_bytes

Socket buffer size for internode communication receive buffer. When not set, buffer size is defined by the OS default (net.ipv4.tcp_rmem).

Default: Not set (OS default)

enable_uuid_sstable_identifiers

Whether newly created SSTables use UUID-based generation identifiers.

Default: true

batchlog_replay_throttle_in_kb

Maximum throttle in KB per second, total, for replaying batchlogs. This is reduced proportionally to the number of nodes in the cluster.

Default: 1024

dynamic_snitch_update_interval_in_ms

How often to perform the more expensive part of host score calculation in milliseconds.

Default: 100

dynamic_snitch_reset_interval_in_ms

How often to reset all host scores in milliseconds, allowing a bad host to possibly recover.

Default: 600000 (10 minutes)

dynamic_snitch_badness_threshold

When the dynamic snitch will prefer other replicas over a pinned host. A value of 0.2 means HCD continues to prefer the static snitch values until the pinned host is 20% worse than the fastest.

Default: 1.0

rpc_keepalive

Whether keepalive is enabled on RPC and native connections.

Default: true

internode_compression

Whether traffic between nodes is compressed.

Available options:

dc (default): Traffic between different datacenters is compressed.
all: All traffic is compressed.
none: Nothing is compressed.

inter_dc_tcp_nodelay

Whether tcp_nodelay is enabled for inter-datacenter communication. Disabling it results in larger (but fewer) network packets being sent, reducing TCP overhead at the cost of increasing latency for cross-datacenter responses.

Default: false

tracetype_query_ttl

TTL in seconds for query trace types used during logging.

Default: 86400 (1 day)

tracetype_repair_ttl

TTL in seconds for repair trace types used during logging of the repair process.

Default: 604800 (7 days)

windows_timer_interval

The default Windows kernel timer and scheduling resolution in milliseconds. Lowering this value on Windows can provide tighter latency and better throughput.

Default: 1

max_value_size_in_mb

The maximum size of any value in SSTables up to a maximum of 2048 MB. If any value exceeds this threshold, HCD marks the SSTables as corrupted.

Default size is the same as the default native protocol frame limit native_transport_max_frame_size_in_mb.

Default: 256

trickle_fsync

Whether flushing of SSTable portions is enabled when trickle_fsync_interval_in_kb is reached. This minimizes sudden flushing of dirty buffers, which can impact read latencies.

Recommended for use with SSDs which can handle more frequent calls to fsync(), but may be detrimental to slow HDDs.

Default: true

trickle_fsync_interval_in_kb

Threshold to trigger a flush when trickle_fsync is enabled.

Default: 10240 (10 MB)

cross_node_timeout

Whether operation timeout information is exchanged between nodes to accurately measure request timeouts. If disabled, replicas assume that requests were forwarded to them instantly by the coordinator.

Default: false

slow_query_log_timeout_in_ms

How long in milliseconds before a node logs slow queries. SELECT queries that take longer than this timeout generate an aggregated log message. Set to 0 to disable slow query logging.

Default: 500

read_request_timeout_in_ms: How long the coordinator should wait for read operations to complete in milliseconds. Lowest acceptable value is 10 ms.

Default: 5000

range_request_timeout_in_ms: How long the coordinator should wait for seq or index scans to complete in milliseconds. Lowest acceptable value is 10 ms.

Default: 10000

write_request_timeout_in_ms: How long the coordinator should wait for writes to complete in milliseconds. Lowest acceptable value is 10 ms.

Default: 2000

counter_write_request_timeout_in_ms: How long the coordinator should wait for counter writes to complete in milliseconds. Lowest acceptable value is 10 ms.

Default: 5000

cas_contention_timeout_in_ms: How long a coordinator should continue to retry a CAS operation that contends with other proposals for the same row in milliseconds. Lowest acceptable value is 10 ms.

Default: 1000

truncate_request_timeout_in_ms: How long the coordinator should wait for truncates to complete in milliseconds. Lowest acceptable value is 10 ms.

Default: 60000
request_timeout_in_ms: The default timeout for other, miscellaneous operations in milliseconds. Lowest acceptable value is 10 ms.

Default: 10000
internode_tcp_connect_timeout_in_ms: The amount of time in milliseconds to wait for internode TCP connections to establish.

Default: 2000
internode_tcp_user_timeout_in_ms: The amount of time in milliseconds unacknowledged data is allowed on a connection before discarding the connection. Only supported on Linux + epoll. Set to 0 to use the OS default.

Default: 30000
internode_streaming_tcp_user_timeout_in_ms: The amount of time in milliseconds unacknowledged data is allowed on a streaming connection.

Default: 300000 (5 minutes)
internode_application_send_queue_capacity_in_bytes: Per-link capacity of messages queued for delivery to other nodes.

Default: 4194304 (4 MiB)
internode_application_send_queue_reserve_endpoint_capacity_in_bytes: Per-endpoint reserve capacity for messages queued for delivery to a single node.

Default: 134217728 (128 MiB)
internode_application_send_queue_reserve_global_capacity_in_bytes: Global limit imposed on messages queued for delivery to all nodes in the cluster.

Default: 536870912 (512 MiB)
internode_application_receive_queue_capacity_in_bytes: Per-link capacity of messages queued on arrival from other nodes.

Default: 4194304 (4 MiB)
internode_application_receive_queue_reserve_endpoint_capacity_in_bytes: Per-endpoint reserve capacity for messages queued on arrival from a single node.

Default: 134217728 (128 MiB)
internode_application_receive_queue_reserve_global_capacity_in_bytes: Global limit imposed on messages queued on arrival from all nodes in the cluster.

Default: 536870912 (512 MiB)
ideal_consistency_level: Track a metric per keyspace indicating whether replication achieved the ideal consistency level for writes without timing out.

Default: Not set
automatic_sstable_upgrade: Whether SSTables are automatically upgraded after a software upgrade. If there is no ordinary compaction to do, the oldest non-upgraded SSTable gets upgraded to the latest version.

Default: false
max_concurrent_automatic_sstable_upgrades: Limit the number of concurrent automatic SSTable upgrades.

Default: 1
corrupted_tombstone_strategy: Validate tombstones on reads and compaction. Can be disabled, warn, or exception.

Default: disabled
diagnostic_events_enabled: Whether diagnostic events are emitted containing details on internal state and temporal relationships across events, accessible by clients via JMX.

Default: false
repaired_data_tracking_for_range_reads_enabled: Whether tracking of repaired state is enabled during range reads and comparison between replicas.

Default: false
repaired_data_tracking_for_partition_reads_enabled: Whether tracking of repaired state is enabled during partition reads and comparison between replicas when the consistency level is greater than ONE and a digest mismatch occurs.

Default: false
report_unconfirmed_repaired_data_mismatches: Whether unconfirmed repaired data mismatches are reported. When false, only confirmed mismatches are reported. When true, a separate metric for unconfirmed mismatches is also recorded.

Default: false
table_count_warn_threshold: Threshold for total number of tables in the cluster above which a client warning is sent when creating a table.

Default: 150
keyspace_count_warn_threshold: Threshold for total number of keyspaces in the cluster above which a client warning is sent when creating a keyspace.

Default: 40
aggregation_subpage_size_in_kb: Page size in KB used internally by aggregation queries to limit memory used when there is a lot of data to aggregate.

Default: 2048

Security properties

Configure authentication, authorization, and role management.

The security properties in cassandra.yaml control how HCD handles user authentication, authorization, and data encryption. These settings are crucial for securing your cluster in production environments.

Authentication properties

authenticator

The authentication backend that implements IAuthenticator to identify users.

HCD provides several authentication options:

AllowAllAuthenticator: Performs no authentication checks. Use this to disable authentication. DataStax does not recommend this for production environments.
PasswordAuthenticator: Relies on username/password pairs stored in the system_auth.roles table.
AdvancedAuthenticator: Allows multiple authentication schemes including internal, OIDC, and LDAP. For LDAP configuration, see Define an LDAP scheme.

Default: AllowAllAuthenticator

If using PasswordAuthenticator, you must also use CassandraRoleManager for role management. Increase the system_auth keyspace replication factor when using authentication.

Authorization properties

authorizer

The authorization backend that implements IAuthorizer to limit access and provide permissions.

HCD provides several authorization options:

AllowAllAuthorizer: Allows any action to any user. Use this to disable authorization. DataStax does not recommend this for production environments.
CassandraAuthorizer: Stores permissions in the system_auth.role_permissions table.
AdvancedAuthorizer: Checks if roles have authorization permissions to access resources.

Default: AllowAllAuthorizer

If using CassandraAuthorizer, increase the system_auth keyspace replication factor.

Role management properties

role_manager

The role management backend that implements IRoleManager to maintain grants and memberships between roles.

HCD provides several role management options:

CassandraRoleManager: Stores role data in the system_auth keyspace.
AdvancedRoleManager: Fetches roles from internal Cassandra tables and/or external servers.

Default: CassandraRoleManager

Most IRoleManager functions require an authenticated login. If the configured IAuthenticator doesn’t implement authentication, most functionality will be unavailable.

Network authorization properties

network_authorizer

The network authorization backend that implements INetworkAuthorizer to restrict user access to certain datacenters.

HCD provides several network authorization options:

AllowAllNetworkAuthorizer: Allows access to any datacenter to any user.
CassandraNetworkAuthorizer: Stores permissions in the system_auth.network_permissions table.

Default: AllowAllNetworkAuthorizer

internode_authenticator

Internode authentication backend implementing IInternodeAuthenticator, used to allow or disallow connections from peer nodes.

Default: org.apache.cassandra.auth.AllowAllInternodeAuthenticator

Client encryption properties

client_encryption_options

Configure client-to-server encryption settings.

client_encryption_options:
    enabled: false                    # Enable client-to-server encryption
    optional: true                    # Allow encrypted and unencrypted connections
    keystore: conf/.keystore         # Path to keystore file
    keystore_password: cassandra     # Keystore password
    require_client_auth: false        # Verify client certificates
    truststore: conf/.truststore     # Path to truststore file
    truststore_password: cassandra   # Truststore password
    protocol: TLS                     # SSL/TLS protocol
    store_type: JKS                   # Keystore type
    cipher_suites: [                 # Supported cipher suites
        TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
        TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
        TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    ]

The default configuration is insecure. Generate proper keystores and truststores before enabling encryption in production.

Server encryption properties

server_encryption_options

Configure server-to-server internode encryption settings.

server_encryption_options:
    internode_encryption: none        # Encryption scope: none, dc, rack, or all
    optional: true                    # Allow encrypted and unencrypted connections
    keystore: conf/.keystore         # Path to keystore file
    keystore_password: cassandra     # Keystore password
    require_client_auth: false        # Verify peer server certificates
    truststore: conf/.truststore     # Path to truststore file
    truststore_password: cassandra   # Truststore password
    require_endpoint_verification: false  # Verify hostname in certificate
    enable_legacy_ssl_storage_port: false # Enable legacy SSL storage port

Encryption scope options:

none: Do not encrypt outgoing connections
dc: Encrypt connections to peers in other datacenters, but not within datacenters
rack: Encrypt connections to peers in other racks, but not within racks
all: Always use encrypted connections

The default configuration is insecure. Generate proper keystores and truststores before enabling encryption in production.

Transparent data encryption properties

transparent_data_encryption_options

Configure transparent data encryption (TDE) for data at rest.

transparent_data_encryption_options:
    enabled: false                    # Enable transparent data encryption
    chunk_length_kb: 64              # Encryption chunk size
    cipher: AES/CBC/PKCS5Padding     # Encryption cipher
    key_alias: testing:1             # Key alias for encryption
    iv_length: 16                    # CBC IV length for AES
    key_provider:
      - class_name: org.apache.cassandra.security.JKSKeyProvider
        parameters:
          - keystore: conf/.keystore
            keystore_password: cassandra
            store_type: JCEKS
            key_password: cassandra

To use the HCD TDE feature, make sure the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy is enabled. In JDK 8u161 and later, this is enabled by default.

This feature supports encryption for commit log and hints files.

Audit logging properties

audit_logging_options

Configure audit logging to track CQL commands and authentication events.

audit_logging_options:
    enabled: false                    # Enable audit logging
    logger:
      - class_name: BinAuditLogger   # Audit logger implementation
    audit_logs_dir:                  # Directory for audit logs
    included_keyspaces:              # Keyspaces to audit
    excluded_keyspaces: system, system_schema, system_virtual_schema
    included_categories:             # Categories to audit
    excluded_categories:             # Categories to exclude
    included_users:                  # Users to audit
    excluded_users:                  # Users to exclude
    roll_cycle: HOURLY              # Log roll cycle
    block: true                      # Block on log write
    max_queue_weight: 268435456     # Max queue weight (256 MiB)
    max_log_size: 17179869184       # Max log size (16 GiB)
    archive_command:                 # Archive command
    max_archive_retries: 10         # Max archive retries

Security cache properties

roles_validity_in_ms: Validity period for roles cache in milliseconds.

HCD caches granted roles for authenticated sessions. After this period, they become eligible for async reload.

Default: 2000

Set to 0 to disable caching entirely.

permissions_validity_in_ms: Validity period for permissions cache in milliseconds.

Default: 2000

Set to 0 to disable caching.

credentials_validity_in_ms: Validity period for credentials cache in milliseconds.

HCD tightly couples this cache to the PasswordAuthenticator implementation.

Default: 2000

Set to 0 to disable caching.

roles_update_interval_in_ms: Refresh interval for roles cache in milliseconds.

After this interval, cache entries become eligible for refresh.

Default: Same value as roles_validity_in_ms

permissions_update_interval_in_ms: Refresh interval for permissions cache in milliseconds.

After this interval, cache entries become eligible for refresh.

Default: Same value as permissions_validity_in_ms

credentials_update_interval_in_ms: Refresh interval for credentials cache in milliseconds.

After this interval, cache entries become eligible for refresh.

Default: Same value as credentials_validity_in_ms

User-defined functions (UDF)

Configure user-defined functions (UDFs) that allow custom logic to be executed within the database.

enable_user_defined_functions: Whether user-defined functions (UDFs) are enabled on this node.

As of Cassandra 3.0, the system has a sandbox in place that should prevent execution of malicious code.

Default: false

enable_scripted_user_defined_functions: Whether scripted UDFs (JavaScript UDFs) are enabled.

Java UDFs are always enabled if enable_user_defined_functions is true. Enable this option to use UDFs with "language javascript" or any custom JSR-223 provider. This option has no effect if enable_user_defined_functions is false.

Default: false

Memory leak detection settings

Configure garbage collection monitoring and memory leak detection thresholds.

The following properties are commented out in the default cassandra.yaml configuration and are not active by default. Uncomment and configure them as needed for your environment.

gc_log_threshold_in_ms: HCD logs GC pauses greater than this threshold at INFO level.

This threshold can be adjusted to minimize logging if necessary.

Default: 200 ms (commented out in default configuration)

gc_warn_threshold_in_ms: HCD logs GC pauses greater than this threshold at WARN level.

Adjust the threshold based on your application throughput requirements. Setting to 0 deactivates the feature.

Default: 1000 ms (commented out in default configuration)

max_value_size_in_mb: Maximum size of any value in SSTables. Safety measure to detect SSTable corruption early.

Any value size larger than this threshold will result in HCD marking an SSTable as corrupted. This should be positive and less than 2048.

Default: 256 MB (commented out in default configuration)

Guardrails

Guardrails are system limits that ensure high availability and optimal performance of the database. They help prevent operations that could cause performance issues or system instability. For more information, see HCD guardrails.

Configure these properties in the guardrails section of cassandra.yaml:

guardrails:
  # Reads
  tombstone_warn_threshold: 1000
  tombstone_failure_threshold: 100000
  page_size_failure_threshold_in_kb: -1
  in_select_cartesian_product_failure_threshold: -1
  partition_keys_in_select_failure_threshold: -1

  # Writes
  batch_size_warn_threshold_in_kb: 64
  batch_size_fail_threshold_in_kb: 640
  unlogged_batch_across_partitions_warn_threshold: 10
  column_value_size_failure_threshold_in_kb: -1
  write_consistency_levels_disallowed:
  read_before_write_list_operations_enabled: true
  user_timestamps_enabled: true

  # Schema
  columns_per_table_failure_threshold: -1
  fields_per_udt_failure_threshold: -1
  collection_size_warn_threshold_in_kb: -1
  items_per_collection_warn_threshold: -1
  tables_warn_threshold: -1
  tables_failure_threshold: -1
  table_properties_disallowed:

  # Indexes
  secondary_index_per_table_failure_threshold: -1
  sai_indexes_per_table_failure_threshold: 10
  sai_indexes_total_failure_threshold: 100
  materialized_view_per_table_failure_threshold: -1

  # Disk usage thresholds
  partition_size_warn_threshold_in_mb: 100
  disk_usage_percentage_warn_threshold: -1
  disk_usage_percentage_failure_threshold: -1
  disk_usage_max_disk_size_in_gb: -1

replica_filtering_protection

Configuration block for replica filtering protection. This mechanism materializes replica results on-heap at the coordinator during filtering or secondary index queries at consistency levels above ONE or LOCAL_ONE to limit the damage severely out-of-date replicas can cause.

replica_filtering_protection:
  cached_rows_warn_threshold: 2000
  cached_rows_fail_threshold: 32000

cached_rows_warn_threshold: The per-query threshold at which a warning will be logged. Default: 2000
cached_rows_fail_threshold: The per-query threshold at which the query will fail. Default: 32000

tombstone_warn_threshold

Log a warning when scanning more tombstones than this threshold.

When executing a scan, within or across a partition, Cassandra keeps tombstones in memory to return them to the coordinator. With workloads that generate many tombstones, this can cause performance problems and even exhaust the server heap.

Default: 1000

tombstone_failure_threshold

Fail queries that scan more tombstones than this threshold.

Default: 100000

page_size_failure_threshold_in_kb

Failure threshold to prevent providing larger paging by bytes than threshold, also served as a hard paging limit when paging by rows is used.

Default: -1 (disabled)

in_select_cartesian_product_failure_threshold

Failure threshold to prevent IN query creating size of cartesian product exceeding threshold.

Example: "a in (1,2,…10) and b in (1,2…10)" results in cartesian product of 100.

Default: -1 (disabled)

partition_keys_in_select_failure_threshold

Failure threshold to prevent IN query containing more partition keys than threshold.

Default: -1 (disabled)

batch_size_warn_threshold_in_kb

Log WARN on any multiple-partition batch size that exceeds this value.

Use caution when increasing this threshold as it can lead to node instability.

Default: 64 KB

batch_size_fail_threshold_in_kb

Fail any multiple-partition batch that exceeds this value.

The calculated default is 640 KB (10x warn threshold).

Default: 640 KB

unlogged_batch_across_partitions_warn_threshold

Log WARN on any batches not of type LOGGED that span across more partitions than this limit.

Default: 10

column_value_size_failure_threshold_in_kb

Failure threshold to prevent writing large column values into HCD.

Default: -1 (disabled)

write_consistency_levels_disallowed

Prevents queries with provided consistency levels.

Default: All consistency levels are allowed.

read_before_write_list_operations_enabled

Whether to allow read-before-write operations, such as setting list element by index or removing list element by index.

Note: Lightweight Transactions (LWT) is always allowed.

Default: true

user_timestamps_enabled

Whether to allow user-provided timestamps in write requests.

Default: true

columns_per_table_failure_threshold

Failure threshold to prevent creating more columns per table than threshold.

Default: -1 (disabled)

fields_per_udt_failure_threshold

Failure threshold to prevent creating more fields in user-defined-type than threshold.

Default: -1 (disabled)

collection_size_warn_threshold_in_kb

Warning threshold to warn when encountering larger size of collection data than threshold.

Default: -1 (disabled)

items_per_collection_warn_threshold

Warning threshold to warn when encountering more elements in collection than threshold.

Default: -1 (disabled)

tables_warn_threshold

Warning threshold to warn when creating more tables than threshold.

Default: -1 (disabled)

tables_failure_threshold

Failure threshold to prevent creating more tables than threshold.

Default: -1 (disabled)

table_properties_disallowed

Prevents creating tables with provided configurations.

Default: All properties are allowed

secondary_index_per_table_failure_threshold

Failure threshold to prevent creating more secondary indexes per table than threshold. Does not apply to CUSTOM INDEX StorageAttachedIndex.

Default: -1 (disabled)

sai_indexes_per_table_failure_threshold

Failure threshold for number of StorageAttachedIndex per table. Only applies to CUSTOM INDEX StorageAttachedIndex.

Default: 10

sai_indexes_total_failure_threshold

Failure threshold for total number of StorageAttachedIndex across all keyspaces. Only applies to CUSTOM INDEX StorageAttachedIndex.

Default: 100

materialized_view_per_table_failure_threshold

Failure threshold to prevent creating more materialized views per table than threshold.

Default: -1 (disabled)

partition_size_warn_threshold_in_mb

Log a warning when compacting partitions larger than this value.

Default: 100 MB

disk_usage_percentage_warn_threshold

Warning threshold to warn when local disk usage exceeds threshold. Valid values: 1, 100.

Default: -1 (disabled)

disk_usage_percentage_failure_threshold

Failure threshold to reject write requests if replica disk usage exceeds threshold. Valid values: 1, 100.

Default: -1 (disabled)

disk_usage_max_disk_size_in_gb

Allows configuring max disk size of data directories when calculating thresholds for disk_usage_percentage_warn_threshold and disk_usage_percentage_failure_threshold.

Valid values: (1, max available disk size of all data directories).

Default: -1 (disabled and use the physically available disk size of data directories during calculations)

emulate_dbaas_defaults

This property isn’t set in the guardrails section, but, if enabled, will overwrite defaults for most guardrails settings.

When enabled, modifies defaults to match those used by DataStax Astra DB, including stricter guardrails and restricted consistency levels. These defaults are based on the Astra DB guardrails at the time of your HCD version’s release, and they are a best-effort emulation of those defaults. Because Astra DB is a managed service, some defaults are unable to be emulated in a self-managed environment.

When enabled on any node, all nodes in the cluster must use this setting.

Default: false

cassandra.yaml configuration file

Syntax

Quick start properties

Default directories

Data directory configuration

Commonly used properties

Common initialization properties

Advanced initialization properties

Common compaction settings

Memtable settings

Common automatic backup settings

Performance tuning

Commit log settings

Change Data Capture (CDC) settings

Compaction settings

Cache and index settings

Streaming settings

Advanced properties

Security properties

Authentication properties

Authorization properties

Role management properties

Network authorization properties

Client encryption properties

Server encryption properties

Transparent data encryption properties

Audit logging properties

Security cache properties

User-defined functions (UDF)

Memory leak detection settings

Guardrails

Was this helpful?

Give Feedback