cassandra.yaml configuration file

The cassandra.yaml file is the main configuration file for DataStax Enterprise. The dse.yaml file is the primary configuration file for security, DSE Search, DSE Graph, and DSE Analytics.

Syntax

For the properties in each section, the parent setting has zero spaces. Each child entry requires at least two spaces. Adhere to the YAML syntax and retain the spacing.

Organization

The configuration properties are grouped into the following sections:

• Quick start

  The minimal properties needed for configuring a cluster.

• Default directories

  If you have changed any of the default directories during installation, set these properties to the new locations. Make sure you have root access.

• Commonly used

  Properties most frequently used when configuring DataStax Enterprise.

• Performance tuning

  Tuning performance and system resource utilization, including commit log, compaction, memory, disk I/O, CPU, reads, and writes.

• Advanced

  Properties for advanced users or properties that are less commonly used.

• Security

  DSE Unified Authentication provides authentication, authorization, and role management.

• Continuous paging options Properties configure memory, threads, and duration when pushing pages continuously to the client.
• Memory leak detection settings Properties configure memory leak detection.

Quick start properties

The minimal properties needed for configuring a cluster.

cluster_name: 'Test Cluster'
listen_address: localhost
# listen_interface: wlan0
# listen_interface_prefer_ipv6: false

cluster_name

The name of the cluster. This setting prevents nodes in one logical cluster from joining another. All nodes in a cluster must have the same value.

Default: 'Test Cluster'

listen_address

The IP address or hostname that the database binds to for connecting this node to other nodes.

Warning:

• Never set listen_address to 0.0.0.0.
• Set listen_address or listen_interface, do not set both.

Default: localhost

listen_interface

The interface that the database binds to for connecting to other nodes. Interfaces must correspond to a single address. IP aliasing is not supported.

Warning: Set listen_address or listen_interface, not both.

Default: commented out (wlan0)

listen_interface_prefer_ipv6

Use IPv4 or IPv6 when interface is specified by name.

• false - use first IPv4 address.
• true - use first IPv6 address.

When only a single address is used, that address is selected without regard to this setting.

Default: commented out (false)

Default directories

data_file_directories:
     - /var/lib/cassandra/data
commitlog_directory: /var/lib/cassandra/commitlog
cdc_raw_directory: /var/lib/cassandra/cdc_raw
hints_directory: /var/lib/cassandra/hints
saved_caches_directory: /var/lib/cassandra/saved_caches

If you have changed any of the default directories during installation, set these properties to the new locations. Make sure you have root access.

data_file_directories

The directory where table data is stored on disk. The database distributes data evenly across the location, subject to the granularity of the configured compaction strategy. If not set, the directory is $DSE_HOME/data/data.

Tip: For production, DataStax recommends RAID 0 and SSDs.

Default: - /var/lib/cassandra/data

commitlog_directory

The directory where the commit log is stored. If not set, the directory is $DSE_HOME/data/commitlog.

For optimal write performance, place the commit log on a separate disk partition, or ideally on a separate physical device from the data file directories. Because the commit log is append only, a hard disk drive (HDD) is acceptable.

Important: DataStax recommends explicitly setting the location of the DSE Metrics Collector data directory.

When the DSE Metrics Collector is enabled and when the insights_options data dir is not explicitly set in dse.yaml, the default location of the DSE Metrics Collector data directory is the same directory as the commitlog directory.

Default: /var/lib/cassandra/commitlog

cdc_raw_directory

The directory where the change data capture (CDC) commit log segments are stored on flush. DataStax recommends a physical device that is separate from the data directories. If not set, the directory is $DSE_HOME/data/cdc_raw. See Change Data Capture (CDC) logging.

Default: /var/lib/cassandra/cdc_raw

hints_directory

The directory in which hints are stored. If not set, the directory is $CASSANDRA_HOME/data/hints.

Default: /var/lib/cassandra/hints

saved_caches_directory

The directory location where table key and row caches are stored. If not set, the directory is $DSE_HOME/data/saved_caches.

Default: /var/lib/cassandra/saved_caches

Commonly used properties

Properties most frequently used when configuring DataStax Enterprise.

Common initialization properties

commit_failure_policy: stop
prepared_statements_cache_size_mb:
# disk_optimization_strategy: ssd
disk_failure_policy: stop
endpoint_snitch: com.datastax.bdp.snitch.DseSimpleSnitch
seed_provider:
    - org.apache.cassandra.locator.SimpleSeedProvider 
        - seeds: "127.0.0.1"
enable_user_defined_functions: false
enable_scripted_user_defined_functions: false
enable_user_defined_functions_threads: true

commit_failure_policy

Policy for commit disk failures:

• die - Shut down the node and kill the JVM, so the node can be replaced.
• stop - Shut down the node, leaving the node effectively dead, available for inspection using JMX.
• stop_commit - Shut down the commit log, letting writes collect but continuing to service reads.
• ignore - Ignore fatal errors and let the batches fail.

Default: stop

prepared_statements_cache_size_mb

Maximum size of the native protocol prepared statement cache. Change this value only if there are more prepared statements than fit in the cache.

Generally, the calculated default value is appropriate and does not need adjusting. DataStax recommends contacting the DataStax Services team before changing this value.

Note: Specifying a value that is too large results in long running GCs and possibly out-of-memory errors. Keep the value at a small fraction of the heap.

Constantly re-preparing statements is a performance penalty. When not set, the default is automatically calculated to heap / 256 or 10 MB, whichever is greater.

Default: calculated

disk_optimization_strategy

The strategy for optimizing disk reads.

• ssd - solid state disks
• spinning - spinning disks

When commented out, the default is ssd.

Default: commented out (ssd)

disk_failure_policy

Sets how the database responds to disk failure. Recommend settings: stop or best_effort. Valid values:

• die - Shut down gossip and client transports, and kill the JVM for any file system errors or single SSTable errors, so the node can be replaced.
• stop_paranoid - Shut down the node, even for single SSTable errors.
• stop - Shut down the node, leaving the node effectively dead, but available for inspection using JMX.
• best_effort - Stop using the failed disk and respond to requests based on the remaining available SSTables. This setting allows obsolete data at consistency level of ONE.
• ignore - Ignore fatal errors and lets the requests fail; all file system errors are logged but otherwise ignored.

Tip: See Recovering from a single disk failure using JBOD.

Default: stop

endpoint_snitch

A class that implements the IEndpointSnitch interface. The database uses the snitch to locate nodes and route requests.

Important: Use only snitch implementations bundled with DSE.

• DseSimpleSnitch

  Appropriate only for development deployments. Proximity is determined by DSE workload, which places transactional, analytics, and search nodes into their separate datacenters. Does not recognize datacenter or rack information.

• GossipingPropertyFileSnitch

  Recommended for production. Reads rack and datacenter for the local node in cassandra-rackdc.properties file and propagates these values to other nodes via gossip. For migration from the PropertyFileSnitch, uses the cassandra-topology.properties file if it is present.

• PropertyFileSnitch

  Determines proximity by rack and datacenter that are explicitly configured in cassandra-topology.properties file.

• Ec2Snitch

  For EC2 deployments in a single region. Loads region and availability zone information from the Amazon EC2 API. The region is treated as the datacenter, the availability zone is treated as the rack, and uses only private IP addresses. For this reason, Ec2Snitch does not work across multiple regions.

• Ec2MultiRegionSnitch

  Uses the public IP as the broadcast_address to allow cross-region connectivity. This means you must also set seed addresses to the public IP and open the storage_port or ssl_storage_port on the public IP firewall. For intra-region traffic, the database switches to the private IP after establishing a connection.

• RackInferringSnitch

  Proximity is determined by rack and datacenter, which are assumed to correspond to the 3rd and 2nd octet of each node's IP address, respectively. Best used as an example for writing a custom snitch class (unless this happens to match your deployment conventions).

• GoogleCloudSnitch

  Use for deployments on Google Cloud Platform across one or more regions. The region is treated as a datacenter and the availability zones are treated as racks within the datacenter. All communication occurs over private IP addresses within the same logical network.

• CloudstackSnitch

  Use the CloudstackSnitch for Apache Cloudstack environments.

Tip: See .

Default: com.datastax.bdp.snitch.DseSimpleSnitch

seed_provider

The addresses of hosts that are designated as contact points in the cluster. A joining node contacts one of the nodes in the -seeds list to learn the topology of the ring.

Important: Use only seed provider implementations bundled with DSE.

• class_name - The class that handles the seed logic. It can be customized, but this is typically not required.

  Default: org.apache.cassandra.locator.SimpleSeedProvider

• - seeds - A comma delimited list of addresses that are used by gossip for bootstrapping new nodes joining a cluster. If your cluster includes multiple nodes, you must change the list from the default value to the IP address of one of the nodes.

  Default: "127.0.0.1"

  Attention: Making every node a seed node is not recommended because of increased maintenance and reduced gossip performance. Gossip optimization is not critical, but it is recommended to use a small seed list (approximately three nodes per datacenter).

Tip: See Initializing a single datacenter per workload type and Initializing multiple datacenters per workload type.

Default: org.apache.cassandra.locator.SimpleSeedProvider

enable_user_defined_functions

Enables user defined functions (UDFs). UDFs present a security risk, since they are executed on the server side. UDFs are executed in a sandbox to contain the execution of malicious code.

• true - Enabled. Supports Java as the code language. Detect endless loops and unintended memory leaks.
• false - Disabled.

Default: false (disabled)

enable_scripted_user_defined_functions

Enables the use of JavaScript language in UDFs.

• true - Enabled. Allow JavaScript in addition to Java as a code language.
• false - Disabled. Only allow Java as a code language.

Note: If enable_user_defined_functions is false, this setting has no impact.

Default: false

enable_user_defined_functions_threads

Enables asynchronous UDF execution which requires a function to complete before being executed again.

• true - Enabled. Only one instance of a function can run at one time. Asynchronous execution prevents UDFs from running too long or forever and destabilizing the cluster.
• false - Disabled. Allows multiple instances of the same function to run simultaneously. Required to use UDFs within GROUP BY clauses.
  CAUTION: Disabling asynchronous UDF execution implicitly disables the security manager. You must monitor the read timeouts for UDFs that run too long or forever, which can cause the cluster to destabilize.

Default: true

Common compaction settings

compaction_throughput_mb_per_sec: 16
compaction_large_partition_warning_threshold_mb: 100

compaction_throughput_mb_per_sec

The MB per second to throttle compaction for the entire system. The faster the database inserts data, the faster the system must compact in order to keep the SSTable count down.

• 16 to 32 x rate of write throughput in MB/second, recommended value.
• 0 - disable compaction throttling

Tip: See Configuring compaction.

Default: 16

compaction_large_partition_warning_threshold_mb

The partition size threshold before logging a warning.

Default: 100

Common memtable settings

memtable_heap_space_in_mb: 2048
memtable_offheap_space_in_mb: 2048

memtable_heap_space_in_mb

The amount of on-heap memory allocated for memtables. The database uses the total of this amount and the value of memtable_offheap_space_in_mb to set a threshold for automatic memtable flush.

Tip: See memtable_cleanup_threshold and Tuning the Java heap.

Default: calculated 1/4 of heap size (2048)

memtable_offheap_space_in_mb

The amount of off-heap memory allocated for memtables. The database uses the total of this amount and the value of memtable_heap_space_in_mb to set a threshold for automatic memtable flush.

Tip: See memtable_cleanup_threshold and Tuning the Java heap.

Default: calculated 1/4 of heap size (2048)

Common automatic backup settings

incremental_backups: false
snapshot_before_compaction: false

incremental_backups

Enables incremental backups.

• true - Enable incremental backups to create a hard link to each SSTable flushed or streamed locally in a backups subdirectory of the keyspace data. Incremental backups enable storing backups off site without transferring entire snapshots.
  Important: The database does not automatically clear incremental backup files. DataStax recommends setting up a process to clear incremental backup hard links each time a new snapshot is created.
• false - Do not enable incremental backups.

Tip: See Enabling incremental backups.

Default: false

snapshot_before_compaction

Whether to take a snapshot before each compaction. A snapshot is useful to back up data when there is a data format change.

Important: Be careful using this option, the database does not clean up older snapshots automatically.

Tip: See Configuring compaction.

Default: false

snapshot_before_dropping_column

When enabled, every time the user drops a column/columns from a table, a snapshot is created on each node in the cluster before the change in schema is applied. Those snapshots have the same name on each node. For example: auto-snapshot_drop-column-columnname_20200515143511000. The name includes the name of the dropped column and the timestamp (UTC) when the column was dropped.

Important: The database does not automatically clear incremental backup files. DataStax recommends setting up a process to clear incremental backup hard links each time a new snapshot is created.

Default: false

Performance tuning properties

Tuning performance and system resource utilization, including commit log, compaction, memory, disk I/O, CPU, reads, and writes.

• Commit log settings
• Lightweight transactions (LWT) settings
• Change-data-capture (CDC) space settings
• Common compaction settings
• Common memtable settings
• Cache and index settings
• Streaming settings

Commit log settings

commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
# commitlog_sync_group_window_in_ms: 1000
# commitlog_sync_batch_window_in_ms: 2  //deprecated
commitlog_segment_size_in_mb: 32
# commitlog_total_space_in_mb: 8192
# commitlog_compression:
#   - class_name: LZ4Compressor
#     parameters:
#         -

commitlog_sync

Commit log synchronization method:

• periodic - Send ACK signal for writes immediately. Commit log is synced every commitlog_sync_period_in_ms.
• group - Send ACK signal for writes after the commit log has been flushed to disk. Wait up to commitlog_sync_group_window_in_ms between flushes.
• batch - Send ACK signal for writes after the commit log has been flushed to disk. Each incoming write triggers the flush task.

Default: periodic

commitlog_sync_period_in_ms

Use with commitlog_sync: periodic. Time interval between syncing the commit log to disk. Periodic syncs are acknowledged immediately.

Default: 10000

commitlog_sync_group_window_in_ms

Use with commitlog_sync: group. The time that the database waits between flushing the commit log to disk. DataStax recommends using group instead of batch.

Default: commented out (1000)

commitlog_sync_batch_window_in_ms

Deprecated. Use with commitlog_sync: batch. The maximum length of time that queries may be batched together.

Default: commented out (2)

commitlog_segment_size_in_mb

The size of an individual commitlog file segment. A commitlog segment may be archived, deleted, or recycled after all its data has been flushed to SSTables. This data can potentially include commitlog segments from every table in the system. The default size is usually suitable, but for commitlog archiving you might want a finer granularity; 8 or 16 MB is reasonable.

Restriction:

If you set max_mutation_size_in_kb explicitly, then you must set commitlog_segment_size_in_mb to:

2 * max_mutation_size_in_kb / 1024

The value must be positive and less than 2048.

Tip: See Commit log archive configuration.

Default: 32

max_mutation_size_in_kb

The maximum size of a mutation before the mutation is rejected. Before increasing the commitlog segment size of the commitlog segments, investigate why the mutations are larger than expected. Look for underlying issues with access patterns and data model, because increasing the commitlog segment size is a limited fix. When not set, the default is calculated as (commitlog_segment_size_in_mb * 1024) / 2.

Default: calculated

commitlog_total_space_in_mb

Disk usage threshold for commit logs before triggering the database flushing memtables to disk. If the total space used by all commit logs exceeds this threshold, the database flushes memtables to disk for the oldest commitlog segments to reclaim disk space by removing those log segments from the commit log. This flushing reduces the amount of data to replay on start-up, and prevents infrequently updated tables from keeping commitlog segments indefinitely. If the commitlog_total_space_in_mb is small, the result is more flush activity on less-active tables.

Tip: See Configuring memtable thresholds.

Default for 64-bit JVMs: calculated (8192 or 25% of the total space of the commit log value, whichever is smaller)

Default for 32-bit JVMs:  calculated (32 or 25% of the total space of the commit log value, whichever is smaller )

commitlog_compression

The compressor to use if commit log is compressed. To make changes, uncomment the commitlog_compression section and these options:

# commitlog_compression:
#   - class_name: LZ4Compressor
#     parameters:
#         -

• class_name: LZ4Compressor, Snappy, or Deflate
• parameters: optional parameters for the compressor

When not set, the default compression for the commit log is uncompressed.

Default: commented out

Lightweight transactions (LWT) settings

# concurrent_lw_transactions: 128
# max_pending_lw_transactions: 10000

concurrent_lw_transactions

Maximum number of permitted concurrent lightweight transactions (LWT).

• A higher number might improve throughput if non-contending LWTs are in heavy use, but will use more memory and might be less successful with contention.
• When not set, the default value is 8x the number of TPC cores. This default value is appropriate for most environments.

Default: calculated 8x the number of TPC cores

max_pending_lw_transactions

Maximum number of lightweight transactions (LWT) in the queue before node reports OverloadedException for LWTs.

Default: 10000

Change-data-capture (CDC) space settings

cdc_enabled: false
cdc_total_space_in_mb: 4096
cdc_free_space_check_interval_ms: 250

See also cdc_raw_directory.

cdc_enabled

Enables change data capture (CDC) functionality on a per-node basis. This modifies the logic used for write path allocation rejection.

• true - use CDC functionality to reject mutations that contain a CDC-enabled table if at space limit threshold in cdc_raw_directory.
• false - standard behavior, never reject.

Default: false

cdc_total_space_in_mb

Total space to use for change-data-capture (CDC) logs on disk. If space allocated for CDC exceeds this value, the database throws WriteTimeoutException on mutations, including CDC-enabled tables. A CDCCompactor (a consumer) is responsible for parsing the raw CDC logs and deleting them when parsing is completed.

Default: calculated (4096 or 1/8th of the total space of the drive where the cdc_raw_directory resides)

cdc_free_space_check_interval_ms

Interval between checks for new available space for CDC-tracked tables when the cdc_total_space_in_mb threshold is reached and the CDCCompactor is running behind or experiencing back pressure. When not set, the default is 250.

Default: commented out (250)

Compaction settings

#concurrent_compactors: 1
# concurrent_validations: 0
concurrent_materialized_view_builders: 2
sstable_preemptive_open_interval_in_mb: 50
# pick_level_on_streaming: false

concurrent_compactors

The number of concurrent compaction processes allowed to run simultaneously on a node, not including validation compactions for anti-entropy repair. Simultaneous compactions help preserve read performance in a mixed read-write workload by limiting the number of small SSTables that accumulate during a single long-running compaction. If your data directories are backed by SSDs, increase this value to the number of cores. If compaction running too slowly or too fast, adjust compaction_throughput_mb_per_sec first.

Important: Increasing concurrent compactors leads to more use of available disk space for compaction, because concurrent compactions happen in parallel, especially for STCS. Ensure that adequate disk space is available before increasing this configuration.

Generally, the calculated default value is appropriate and does not need adjusting. DataStax recommends contacting the DataStax Services team before changing this value.

Default: calculated The fewest number of disks or number of cores, with a minimum of 2 and a maximum of 8 per CPU core.

concurrent_validations

Number of simultaneous repair validations to allow. When not set, the default is unbounded. Values less than one are interpreted as unbounded.

Default: commented out (0) unbounded

concurrent_materialized_view_builders

Number of simultaneous materialized view builder tasks allowed to run concurrently. When a view is created, the node ranges are split into (num_processors * 4) builder tasks and submitted to this executor.

Default: 2

sstable_preemptive_open_interval_in_mb

The size of the SSTables to trigger preemptive opens. The compaction process opens SSTables before they are completely written and uses them in place of the prior SSTables for any range previously written. This process helps to smoothly transfer reads between the SSTables by reducing cache churn and keeps hot rows hot.

Important: A low value has a negative performance impact and will eventually cause heap pressure and GC activity. The optimal value depends on hardware and workload.

Default: 50

pick_level_on_streaming

The compaction level for streamed-in SSTables.

• true - streamed-in SSTables of tables using are placed on the same level as the source node. For operational tasks like nodetool refresh or replacing a node, true improves performance for compaction work.
• false - streamed-in SSTables are placed in level 0.

When not set, the default is false.

Default: commented out (false)

Memtable settings

memtable_allocation_type: heap_buffers
# memtable_cleanup_threshold: 0.34
memtable_flush_writers: 4

memtable_allocation_type

The method the database uses to allocate and manage memtable memory.

• heap_buffers - On heap NIO (non-blocking I/O) buffers.
• offheap_buffers - Off heap (direct) NIO buffers.
• offheap_objects - Native memory, eliminating NIO buffer heap overhead.

Default: heap_buffers

memtable_cleanup_threshold

Ratio used for automatic memtable flush.

Generally, the calculated default value is appropriate and does not need adjusting. DataStax recommends contacting the DataStax Services team before changing this value.

When not set, the calculated default is 1/(memtable_flush_writers + 1)

Default: commented out (0.34)

memtable_flush_writers

The number of memtable flush writer threads per disk and the total number of memtables that can be flushed concurrently, generally a combination of compute that is I/O bound. Memtable flushing is more CPU efficient than memtable ingest. A single thread can keep up with the ingest rate of a server on a single fast disk, until the server temporarily becomes I/O bound under contention, typically with compaction. Generally, the default value is appropriate and does not need adjusting for SSDs. However, the recommended default for HDDs: 2.

Default for SSDs: 4

Cache and index settings

column_index_size_in_kb: 16
# file_cache_size_in_mb: 4096
# direct_reads_size_in_mb: 128

column_index_size_in_kb

Granularity of the index of rows within a partition. For huge rows, decrease this setting to improve seek time. Lower density nodes might benefit from decreasing this value to 4, 2, or 1.

Default: 16

file_cache_size_in_mb

DSE 6.0.0-6.0.6: Maximum memory for buffer pooling and SSTable chunk cache. 32 MB is reserved for pooling buffers, the remaining memory is the cache for holding recent or frequently used index pages and uncompressed SSTable chunks. This pool is allocated off heap and is in addition to the memory allocated for heap. Memory is allocated only when needed.

DSE 6.0.7 and later: Buffer pool is split into two pools, this setting defines the maximum memory to use file buffers that are stored in the file cache, also known as chunk cache. Memory is allocated only when needed but is not released. The other buffer pool is direct_reads_size_in_mb.

See Tuning Java Virtual Machine.

Default: calculated (0.5 of -XX:MaxDirectMemorySize)

direct_reads_size_in_mb

DSE 6.0.7 and later: Buffer pool is split into two pools, this setting defines the buffer pool for transient read operations. A buffer is typically used by a read operation and then returned to this pool when the operation is finished so that it can be reused by other operations. The other buffer pool is file_cache_size_in_mb. When not set, the default calculated as 2 MB per TPC core thread, plus 2 MB shared by non-TPC threads, with a maximum value of 128 MB.

Default: calculated

Streaming settings

# stream_throughput_outbound_megabits_per_sec: 200
# inter_dc_stream_throughput_outbound_megabits_per_sec: 200
# streaming_keep_alive_period_in_secs: 300
# streaming_connections_per_host: 1

stream_throughput_outbound_megabits_per_sec

Throttle for the throughput of all outbound streaming file transfers on a node. The database does mostly sequential I/O when streaming data during bootstrap or repair which can saturate the network connection and degrade client (RPC) performance. When not set, the value is 200 Mbps.

Default: commented out (200)

inter_dc_stream_throughput_outbound_megabits_per_sec

Throttle for all streaming file transfers between datacenters, and for network stream traffic as configured with stream_throughput_outbound_megabits_per_sec. When not set, the value is 200 Mbps.

Note: Should be set to a value less than or equal to stream_throughput_outbound_megabits_per_sec since it is a subset of total throughput.

Default: commented out (200)

streaming_keep_alive_period_in_secs

Interval to send keep-alive messages to prevent reset connections during streaming. The stream session fails when a keep-alive message is not received for 2 keep-alive cycles. When not set, the default is 300 seconds (5 minutes) so that a stalled stream times out in 10 minutes.

Default: commented out (300)

streaming_connections_per_host

Maximum number of connections per host for streaming. Increase this value when you notice that joins are CPU-bound, rather than network-bound. For example, a few nodes with large files. When not set, the default is 1.

Default: commented out (1)

Fsync settings

trickle_fsync: true
trickle_fsync_interval_in_kb: 10240

trickle_fsync

When set to true, causes fsync to force the operating system to flush the dirty buffers at the set interval trickle_fsync_interval_in_kb. Enable this parameter to prevent sudden dirty buffer flushing from impacting read latencies. Recommended for use with SSDs, but not with HDDs.

Default: false

trickle_fsync_interval_in_kb

The size of the fsync in kilobytes.

Default: 10240

max_value_size_in_mb

The maximum size of any value in SSTables. SSTables are marked as corrupted when the threshold is exceeded.

Default: 256

Thread Per Core (TPC) parameters

#tpc_cores:
# tpc_io_cores:
io_global_queue_depth: 128

tpc_cores

The number of concurrent CoreThreads. The CoreThreads are the main workers in a DSE 6.x node, and process various asynchronous tasks from their queue. If not set, the default is the number of cores (processors on the machine) minus one. Note that configuring tpc_cores affects the default value for tpc_io_cores.

To achieve optimal throughput and latency, for a given workload, set tpc_cores to half the number of CPUs (minimum) to double the number of CPUs (maximum). In cases where there are a large number of incoming client connections, increasing tpc_cores to more than the default usually results in CoreThreads receiving more CPU time.

Attention: DSE Search workloads only: set tpc_cores to the number of physical CPUs. See Tuning search for maximum indexing throughput.

Default: commented out; defaults to the number of cores minus one.

tpc_io_cores

The subset of tpc_cores that process asynchronous IO tasks. (That is, disk reads.) Must be smaller or equal to tpc_cores. Lower this value to decrease parallel disk IO requests.

Default: commented out; by default, calculated as min(io_global_queue_depth/4, tpc_cores)

io_global_queue_depth

Global IO queue depth used for reads when AIO is enabled, which is the default for SSDs. The optimal queue depth as found with the fio tool for a given disk setup.

Default: 128

NodeSync parameters

nodesync:
    rate_in_kb: 1024

rate_in_kb

The maximum kilobytes per second for data validation on the local node. The optimum validation rate for each node may vary.

Default: 1024

Advanced properties

Properties for advanced users or properties that are less commonly used.

Advanced initialization properties

batch_size_warn_threshold_in_kb: 64
batch_size_fail_threshold_in_kb: 640
unlogged_batch_across_partitions_warn_threshold: 10
# broadcast_address: 1.2.3.4
# listen_on_broadcast_address: false
# initial_token:
# num_tokens: 128
# allocate_tokens_for_local_replication_factor: 3
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
tracetype_query_ttl: 86400
tracetype_repair_ttl: 604800

auto_bootstrap

This setting has been removed from default configuration.

• true - causes new (non-seed) nodes migrate the right data to themselves automatically
• false - When initializing a fresh cluster without data

Tip: See Initializing a DataStax Enterprise cluster.

When not set, the internal default is true.

Default: not present

batch_size_warn_threshold_in_kb

Threshold to log a warning message when any multiple-partition batch size exceeds this value in kilobytes.

CAUTION: Increasing this threshold can lead to node instability.

Default: 64

batch_size_fail_threshold_in_kb

Threshold to fail and log WARN on any multiple-partition batch whose size exceeds this value. The default value is 10X the value of batch_size_warn_threshold_in_kb.

Default: 640

unlogged_batch_across_partitions_warn_threshold

Threshold to log a WARN message on any batches not of type LOGGED that span across more partitions than this limit.

Default: 10

broadcast_address

The public IP address this node uses to broadcast to other nodes outside the network or across regions in multiple-region EC2 deployments. If this property is commented out, the node uses the same IP address or hostname as listen_address. A node does not need a separate broadcast_address in a single-node or single-datacenter installation, or in an EC2-based network that supports automatic switching between private and public communication. It is necessary to set a separate listen_address and broadcast_address on a node with multiple physical network interfaces or other topologies where not all nodes have access to other nodes by their private IP addresses. For specific configurations, see the instructions for listen_address.

Default: listen_address

listen_on_broadcast_address

Enables the node to communicate on both interfaces.

• true - If this node uses multiple physical network interfaces, set a unique IP address for broadcast_address
• false - if this node is on a network that automatically routes between public and private networks, like Amazon EC2 does

Tip: See listen_address.

Default: false

initial_token

The token to start the contiguous range. Set this property for single-node-per-token architecture, in which a node owns exactly one contiguous range in the ring space. Setting this property overrides num_tokens.

If your installation is not using vnodes or this node's num_tokens is set it to 1 or is commented out, you should always set an initial_token value when setting up a production cluster for the first time, and when adding capacity. See Generating tokens.

Use this parameter only with num_tokens (vnodes ) in special cases such as Restoring from a snapshot.

Default: 1 (disabled)

num_tokens

Define virtual node (vnode) token architecture.

Note: All other nodes in the datacenter must have the same token architecture.

• 1 - disable vnodes and use 1 token for legacy compatibility.
• a number between 2 and 128 - the number of token ranges to assign to this virtual node (vnode). A higher value increases the probability that the data and workload are evenly distributed.
  CAUTION: DataStax recommends not using vnodes with DSE Search. However, if you decide to use vnodes with DSE Search, do not use more than 8 vnodes and ensure that allocate_tokens_for_local_replication_factor option in cassandra.yaml is correctly configured for your environment.
  CAUTION: Using vnodes can impact performance for your cluster. DataStax recommends testing the configuration before enabling vnodes in production environments.

  When the token number varies between nodes in a datacenter, the vnode logic assigns a proportional number of ranges relative to other nodes in the datacenter. In general, if all nodes have equal hardware capability, each node should have the same num_tokens value.

Default: 1 (disabled)

To migrate an existing cluster from single node per token range to vnodes, see Enabling virtual nodes on an existing production cluster.

allocate_tokens_for_local_replication_factor

• RF of keyspaces in datacenter - triggers the recommended algorithmic allocation for the RF and num_tokens for this node.

  The allocation algorithm optimizes the workload balance using the target keyspace replication factor. DataStax recommends setting the number of tokens to 8 to distribute the workload with ~10% variance between nodes. The allocation algorithm attempts to choose tokens in a way that optimizes replicated load over the nodes in the datacenter for the specified RF. The load assigned to each node is close to proportional to the number of vnodes.

  Note: The allocation algorithm is supported only for the Murmur3Partitioner and RandomPartitioner partitioners. The Murmur3Partitioner is the default partitioning strategy for new clusters and the right choice for new clusters in almost all cases.
• commented out - uses the random selection algorithm to assign token ranges randomly.
  Note: Over time, loads in a datacenter using the random selection algorithm become unevenly distributed. DataStax recommends using only the allocation algorithm.

Default: commented out (use random selection algorithm)

See Virtual node (vnode) configuration, and for set up instructions see Adding nodes to vnode-enabled cluster or Adding a datacenter to a cluster.

partitioner

The class that distributes rows (by partition key) across all nodes in the cluster. Any IPartitioner may be used, including your own as long as it is in the class path. For new clusters use the default partitioner.

DataStax Enterprise provides the following partitioners for backward compatibility:

• RandomPartitioner
• ByteOrderedPartitioner (deprecated)
• OrderPreservingPartitioner (deprecated)

Important: Use only partitioner implementations bundled with DSE.

Tip: See .

Default: org.apache.cassandra.dht.Murmur3Partitioner

tracetype_query_ttl

TTL for different trace types used during logging of the query process.

Default: 86400

tracetype_repair_ttl

TTL for different trace types used during logging of the repair process.

Default: 604800

Advanced automatic backup setting

auto_snapshot: true

auto_snapshot

Enables snapshots of the data before truncating a keyspace or dropping a table. To prevent data loss, DataStax strongly advises using the default setting. If you set auto_snapshot to false, you lose data on truncation or drop.

Default: true

Global row properties

column_index_cache_size_in_kb: 2
# row_cache_class_name: org.apache.cassandra.cache.OHCProvider
row_cache_size_in_mb: 0
row_cache_save_period: 0
# row_cache_keys_to_save: 100

When creating or modifying tables, you can enable or disable the row cache for that table by setting the caching parameter. Other row cache tuning and configuration options are set at the global (node) level. The database uses these settings to automatically distribute memory for each table on the node based on the overall workload and specific table usage. You can also configure the save periods for these caches globally.

column_index_cache_size_in_kb

(Only applies to BIG format SSTables) Threshold for the total size of all index entries for a partition that the database stores in the partition key cache. If the total size of all index entries for a partition exceeds this amount, the database stops putting entries for this partition into the partition key cache.

Default: 2

row_cache_class_name

The classname of the row cache provider to use. Valid values:

• org.apache.cassandra.cache.OHCProvider - fully off-heap
• org.apache.cassandra.cache.SerializingCacheProvider - partially off-heap, available in earlier releases

Important: Use only row cache provider implementations bundled with DSE.

When not set, the default is org.apache.cassandra.cache.OHCProvider (fully off-heap)

Default: commented out (org.apache.cassandra.cache.OHCProvider)

row_cache_size_in_mb

Maximum size of the row cache in memory. The row cache can save time, but it is space-intensive because it contains the entire row. Use the row cache only for hot rows or static rows. If you reduce the size, you may not get the hottest keys loaded on start up.

• 0 - disable row caching
• MB - Maximum size of the row cache in memory

Default: 0 (disabled)

row_cache_save_period

The number of seconds that rows are kept in cache. Caches are saved to saved_caches_directory. This setting has limited use as described in row_cache_size_in_mb.

Default: 0 (disabled)

row_cache_keys_to_save

The number of keys from the row cache to save. All keys are saved.

Default: commented out (100)

Counter caches properties

counter_cache_size_in_mb:
counter_cache_save_period: 7200
# counter_cache_keys_to_save: 100

Counter cache helps to reduce counter locks' contention for hot counter cells. In case of RF = 1 a counter cache hit causes the database to skip the read before write entirely. With RF > 1 a counter cache hit still helps to reduce the duration of the lock hold, helping with hot counter cell updates, but does not allow skipping the read entirely. Only the local (clock, count) tuple of a counter cell is kept in memory, not the whole counter, so it is relatively cheap.

counter_cache_size_in_mb

When no value is set, the database uses the smaller of minimum of 2.5% of heap or 50 megabytes (MB). If your system performs counter deletes and relies on low gc_grace_seconds, you should disable the counter cache. To disable, set to 0.

Default: calculated

counter_cache_save_period

The time, in seconds, after which the database saves the counter cache (keys only). The database saves caches to saved_caches_directory.

Default: 7200 (2 hours)

counter_cache_keys_to_save

Number of keys from the counter cache to save. When not set, the database saves all keys.

Default: commented out (disabled, saves all keys)

Tombstone settings

tombstone_warn_threshold: 1000
tombstone_failure_threshold: 100000

When executing a scan, within or across a partition, the database must keep tombstones in memory to allow them to return to the coordinator. The coordinator uses tombstones to ensure that other replicas know about the deleted rows. Workloads that generate numerous tombstones may cause performance problems and exhaust the server heap. Adjust these thresholds only if you understand the impact and want to scan more tombstones. You can adjust these thresholds at runtime using the StorageServiceMBean.

tombstone_warn_threshold

The database issues a warning if a query scans more than this number of tombstones.

Default: 1000

tombstone_failure_threshold

The database aborts a query if it scans more than this number of tombstones.

Default: 100000

Network timeout settings

read_request_timeout_in_ms: 5000
range_request_timeout_in_ms: 10000
aggregated_request_timeout_in_ms: 120000
write_request_timeout_in_ms: 2000
counter_write_request_timeout_in_ms: 5000
cas_contention_timeout_in_ms: 1000
truncate_request_timeout_in_ms: 60000
request_timeout_in_ms: 10000
# cross_dc_rtt_in_ms: 0

read_request_timeout_in_ms

Default: 5000. How long the coordinator waits for read operations to complete before timing it out.

range_request_timeout_in_ms

Default: 10000. How long the coordinator waits for sequential or index scans to complete before timing it out.

aggregated_request_timeout_in_ms

How long the coordinator waits for sequential or index scans to complete. Lowest acceptable value is 10 ms. This timeout does not apply to aggregated queries such as SELECT, COUNT(*), MIN(x), and so on.

Default: 120000 (2 minutes)

write_request_timeout_in_ms

How long the coordinator waits for write requests to complete with at least one node in the local datacenter. Lowest acceptable value is 10 ms.

Tip: See .

Default: 2000 (2 seconds)

counter_write_request_timeout_in_ms

How long the coordinator waits for counter writes to complete before timing it out.

Default: 5000 (5 seconds)

cas_contention_timeout_in_ms

How long the coordinator continues to retry a CAS (compare and set) operation that contends with other proposals for the same row. If the coordinator cannot complete the operation within this timespan, it aborts the operation.

Default: 1000 (1 second)

truncate_request_timeout_in_ms

How long the coordinator waits for a truncate (the removal of all data from a table) to complete before timing it out. The long default value allows the database to take a snapshot before removing the data. If auto_snapshot is disabled (not recommended), you can reduce this time.

Default: 60000 (1 minute)

request_timeout_in_ms

The default timeout value for other miscellaneous operations. Lowest acceptable value is 10 ms.

Tip: See .

Default: 10000

cross_dc_rtt_in_ms

How much to increase the cross-datacenter timeout (write_request_timeout_in_ms + cross_dc_rtt_in_ms) for requests that involve only nodes in a remote datacenter. This setting is intended to reduce hint pressure.

Tip: DataStax recommends using LOCAL_* consistency levels (CL) for read and write requests in multi-datacenter deployments to avoid timeouts that may occur when remote nodes are chosen to satisfy the CL, such as QUORUM.

Default: commented out (0)

slow_query_log_timeout_in_ms

Default: 500. How long before a node logs slow queries. Select queries that exceed this value generate an aggregated log message to identify slow queries. To disable, set to 0.

Inter-node settings

storage_port: 7000
cross_node_timeout: false
# internode_send_buff_size_in_bytes:
# internode_recv_buff_size_in_bytes:
internode_compression: dc
inter_dc_tcp_nodelay: false

storage_port

The port for inter-node communication. Follow security best practices, do not expose this port to the internet. Apply firewall rules.

Tip: See .

Default: 7000

cross_node_timeout

Enables operation timeout information exchange between nodes to accurately measure request timeouts. If this property is disabled, the replica assumes any requests are forwarded to it instantly by the coordinator. During overload conditions this means extra time is required for processing already-timed-out requests.

CAUTION: Before enabling this property make sure NTP (network time protocol) is installed and the times are synchronized among the nodes.

Default: false

internode_send_buff_size_in_bytes

The sending socket buffer size, in bytes, for inter-node calls.

Tip: See TCP settings.

The sending socket buffer size and internode_recv_buff_size_in_bytes is limited by net.core.wmem_max. If this property is not set, net.ipv4.tcp_wmem determines the buffer size. For more details run man tcp and refer to:

• /proc/sys/net/core/wmem_max
• /proc/sys/net/core/rmem_max
• /proc/sys/net/ipv4/tcp_wmem
• /proc/sys/net/ipv4/tcp_wmem

Default: not set

internode_recv_buff_size_in_bytes

The receiving socket buffer size in bytes for inter-node calls.

Default: not set

internode_compression

Controls whether traffic between nodes is compressed. Valid values:

• all - Compresses all traffic
• dc - Compresses traffic between datacenters only
• none - No compression.

Default: dc

inter_dc_tcp_nodelay

Enables tcp_nodelay for inter-datacenter communication. When disabled, the network sends larger, but fewer, network packets. This reduces overhead from the TCP protocol itself. However, disabling inter_dc_tcp_nodelay may increase latency by blocking cross datacenter responses.

Default: false

Native transport (CQL Binary Protocol)

start_native_transport: true
native_transport_port: 9042
# native_transport_port_ssl: 9142
# native_transport_max_frame_size_in_mb: 256
# native_transport_max_concurrent_connections: -1
# native_transport_max_concurrent_connections_per_ip: -1
native_transport_address: localhost
# native_transport_interface: eth0
# native_transport_interface_prefer_ipv6: false
# native_transport_broadcast_address: 1.2.3.4
native_transport_keepalive: true

start_native_transport

Enables or disables the native transport server.

Default: true

native_transport_port

The port where the CQL native transport listens for clients. For security reasons, do not expose this port to the internet. Firewall it if needed.

Default: 9042

native_transport_max_frame_size_in_mb

The maximum allowed size of a frame. Frame (requests) larger than this are rejected as invalid.

Default: 256

native_transport_max_concurrent_connections

The maximum number of concurrent client connections.

Default: -1 (unlimited)

native_transport_max_concurrent_connections_per_ip

The maximum number of concurrent client connections per source IP address.

Default: -1 (unlimited)

native_transport_address

When left blank, uses the configured hostname of the node. Unlike the listen_address, this value can be set to 0.0.0.0, but you must set the native_transport_broadcast_address to a value other than 0.0.0.0.

Note: Set native_transport_address OR native_transport_interface, not both.

Default: localhost

native_transport_interface

IP aliasing is not supported.

Note: Set native_transport_address OR native_transport_interface, not both.

Default: eth0

native_transport_interface_prefer_ipv6

Use IPv4 or IPv6 when interface is specified by name.

• false - use first IPv4 address.
• true - use first IPv6 address.

When only a single address is used, that address is selected without regard to this setting.

Default: commented out (false)

native_transport_broadcast_address

Native transport address to broadcast to drivers and other DSE nodes. This cannot be set to 0.0.0.0.

• blank - will be set to the value of native_transport_address
• IP_address - when native_transport_address is set to 0.0.0.0

Default: commented out (1.2.3.4)

native_transport_keepalive

Enables keepalive on native connections.

Default: true

Advanced fault detection settings

# gc_log_threshold_in_ms: 200
# gc_warn_threshold_in_ms: 1000
# otc_coalescing_strategy: DISABLED
# otc_coalescing_window_us: 200
# otc_coalescing_enough_coalesced_messages: 8

gc_log_threshold_in_ms

The threshold for log messages at the INFO level. Adjust to minimize logging.

Default: commented out (200)

gc_warn_threshold_in_ms

Threshold for GC pause. Any GC pause longer than this interval is logged at the WARN level. By default, the database logs any GC pause greater than 200 ms at the INFO level.

Tip: See Configuring logging.

Default: commented out (1000)

otc_coalescing_strategy

Strategy to combine multiple network messages into a single packet for outbound TCP connections to nodes in the same data center. See the DataStax Developer Blog post Performance doubling with message coalescing.

Important: Use only strategy implementations bundled with DSE.

Supported strategies are:

• FIXED
• MOVINGAVERAGE
• TIMEHORIZON
• DISABLED

Default: commented out (DISABLED)

otc_coalescing_window_us

How many microseconds to wait for coalescing messages to nodes in the same datacenter.

• For FIXED strategy - the amount of time after the first message is received before it is sent with any accompanying messages.
• For MOVING average - the maximum wait time and the interval that messages must arrive on average to enable coalescing.

Default: commented out (200)

otc_coalescing_enough_coalesced_messages

The threshold for the number of messages to nodes in the same data center. Do not coalesce messages when this value is exceeded. Should be more than 2 and less than 128.

Default: commented out (8)

seed_gossip_probability

The percentage of time that gossip messages are sent to a seed node during each round of gossip. Decreases the time to propagate gossip changes across the cluster.

Default: 1.0 (100%)

Backpressure settings

back_pressure_enabled: false
back_pressure_strategy:
    - class_name: org.apache.cassandra.net.RateBasedBackPressure
      parameters:
        - high_ratio: 0.90
          factor: 5
          flow: FAST

back_pressure_enabled

Enables the coordinator to apply the specified back pressure strategy to each mutation that is sent to replicas.

Default: false

back_pressure_strategy

To add new strategies, implement org.apache.cassandra.net.BackpressureStrategy and provide a public constructor that accepts a Map<String, Object>.

Important: Use only strategy implementations bundled with DSE.

class_name

The default class_name uses the ratio between incoming mutation responses and outgoing mutation requests.

Default: org.apache.cassandra.net.RateBasedBackPressure

high_ratio

When outgoing mutations are below this value, they are rate limited according to the incoming rate decreased by the factor (described below). When above this value, the rate limiting is increased by the factor.

Default: 0.90

factor

A number between 1 and 10. When backpressure is below high ratio, outgoing mutations are rate limited according to the incoming rate decreased by the given factor; if above high ratio, the rate limiting is increased by the given factor.

Default: 5

flow

The flow speed to apply rate limiting:

• FAST - rate limited to the speed of the fastest replica
• SLOW - rate limit to the speed of the slowest replica

Default: FAST

dynamic_snitch_badness_threshold

The performance threshold for dynamically routing client requests away from a poorly performing node. Specifically, it controls how much worse a poorly performing node has to be before the dynamic snitch prefers other replicas. A value of 0.2 means the database continues to prefer the static snitch values until the node response time is 20% worse than the best performing node. Until the threshold is reached, incoming requests are statically routed to the closest replica as determined by the snitch.

Default: 0.1

dynamic_snitch_reset_interval_in_ms

Time interval after which the database resets all node scores. This allows a bad node to recover.

Default: 600000

dynamic_snitch_update_interval_in_ms

The time interval, in milliseconds, between the calculation of node scores. Because score calculation is CPU intensive, be careful when reducing this interval.

Default: 100

Hinted handoff options

hinted_handoff_enabled: true
# hinted_handoff_disabled_datacenters:
#    - DC1
#    - DC2
max_hint_window_in_ms: 10800000 # 3 hours
hinted_handoff_throttle_in_kb: 1024
max_hints_delivery_threads: 2
hints_directory: /var/lib/cassandra/hints
hints_flush_period_in_ms: 10000
max_hints_file_size_in_mb: 128
#hints_compression:
#   - class_name: LZ4Compressor
#     parameters:
#         -
batchlog_replay_throttle_in_kb: 1024
# batchlog_endpoint_strategy: random_remote

hinted_handoff_enabled

Enables or disables hinted handoff. A hint indicates that the write needs to be replayed to an unavailable node. The database writes the hint to a hints file on the coordinator node.

• false - do not enable hinted handoff
• true - globally enable hinted handoff, except for datacenters specified for hinted_handoff_disabled_datacenters

Default: true

hinted_handoff_disabled_datacenters

A blacklist of datacenters that will not perform hinted handoffs. To disable hinted handoff on a certain datacenter, add its name to this list.

Default: commented out

max_hint_window_in_ms

Maximum amount of time during which the database generates hints for an unresponsive node. After this interval, the database does not generate any new hints for the node until it is back up and responsive. If the node goes down again, the database starts a new interval. This setting can prevent a sudden demand for resources when a node is brought back online and the rest of the cluster attempts to replay a large volume of hinted writes.

Tip: See .

Default: 10800000 (3 hours)

hinted_handoff_throttle_in_kb

Maximum amount of traffic per delivery thread in kilobytes per second. This rate reduces proportionally to the number of nodes in the cluster. For example, if there are two nodes in the cluster, each delivery thread uses. the maximum rate. If there are three, each node throttles to half of the maximum, since the two nodes are expected to deliver hints simultaneously.

Note: When applying this limit, the calculated hint transmission rate is based on the uncompressed hint size, even if internode_compression or hints_compression is enabled.

Default: 1024

hints_flush_period_in_ms

The time, in milliseconds, to wait before flushing hints from internal buffers to disk.

Default: 10000

max_hints_delivery_threads

Number of threads the database uses to deliver hints. In multiple datacenter deployments, consider increasing this number because cross datacenter handoff is generally slower.

Default: 2

max_hints_file_size_in_mb

The maximum size for a single hints file, in megabytes.

Default: 128

hints_compression

The compressor for hint files. Supported compressors: LZ, Snappy, and Deflate. When not set, the database does not compress hints files.

Default: LZ4Compressor

batchlog_replay_throttle_in_kb

Total maximum throttle, in KB per second, for replaying hints. Throttling is reduced proportionally to the number of nodes in the cluster

Default: 1024

batchlog_endpoint_strategy

Strategy to choose the batchlog storage endpoints.

• random_remote - Default, purely random. Prevents the local rack, if possible. Same behavior as earlier releases.
• dynamic_remote - Uses DynamicEndpointSnitch to select batchlog storage endpoints. Prevents the local rack, if possible. This strategy offers the same availability guarantees as random_remote, but selects the fastest endpoints according to the DynamicEndpointSnitch. DynamicEndpointSnitch tracks reads but not writes. Write-only, or mostly-write, workloads might not benefit from this strategy. Note: this strategy will fall back to random_remote if dynamic_snitch is not enabled.
• dynamic - Mostly the same as dynamic_remote, except that local rack is not excluded, which offers lower availability guarantee than random_remote or dynamic_remote. Note: this strategy will fall back to random_remote if dynamic_snitch is not enabled.

Default: random_remote

Security properties

DSE Advanced Security fortifies DataStax Enterprise (DSE) databases against potential harm due to deliberate attack or user error. Configuration properties include authentication and authorization, permissions, roles, encryption of data in-flight and at-rest, and data auditing. DSE Unified Authentication provides authentication, authorization, and role management. Enabling DSE Unified Authentication requires additional configuration in dse.yaml, see .

authenticator: com.datastax.bdp.cassandra.auth.DseAuthenticator
# internode_authenticator: org.apache.cassandra.auth.AllowAllInternodeAuthenticator
authorizer: com.datastax.bdp.cassandra.auth.DseAuthorizer
role_manager: com.datastax.bdp.cassandra.auth.DseRoleManager
system_keyspaces_filtering: false
roles_validity_in_ms: 120000
# roles_update_interval_in_ms: 120000
permissions_validity_in_ms: 120000
# permissions_update_interval_in_ms: 120000

authenticator

The authentication backend. The only supported authenticator is DseAuthenticator for external authentication with multiple authentication schemes such as Kerberos, LDAP, and internal authentication. Authenticators other than DseAuthenticator are deprecated and not supported. Some security features might not work correctly if other authenticators are used. See authentication_options in dse.yaml.

Important: Use only authentication implementations bundled with DSE.

Default: com.datastax.bdp.cassandra.auth.DseAuthenticator

internode_authenticator

Internode authentication backend to enable secure connections from peer nodes.

Important: Use only authentication implementations bundled with DSE.

Default: org.apache.cassandra.auth.AllowAllInternodeAuthenticator

authorizer

The authorization backend. Authorizers other than DseAuthorizer are not supported. DseAuthorizer supports enhanced permission management of DSE-specific resources. Authorizers other than DseAuthorizer are deprecated and not supported. Some security features might not work correctly if other authorizers are used. See Authorization options in dse.yaml.

Important: Use only authorization implementations bundled with DSE.

Default: com.datastax.bdp.cassandra.auth.DseAuthorizer

system_keyspaces_filtering

Enables system keyspace filtering so that users can access and view only schema information for rows in the system and system_schema keyspaces to which they have access. When system_keyspaces_filtering is set to true:

• Data in the system.local and system.peers tables are visible
• Data in the following tables of the system keyspace are filtered based on the role's DESCRIBE privileges for keyspaces; only rows for appropriate keyspaces will be displayed in:
  • size_estimates
  • sstable_activity
  • built_indexes
  • built_views
  • available_ranges
  • view_builds_in_progress
• Data in all tables in the system_schema keyspace are filtered based on a role's DESCRIBE privileges for keyspaces stored in the system_schema tables.
• Read operations against other tables in the system keyspace are denied

Attention: Security requirements and user permissions apply. Enable this feature only after appropriate user permissions are granted. You must grant the DESCRIBE permission to role on any keyspaces stored in the system keyspaces. If you do not grant the permission, you will see an error that states the keyspace is not found.

GRANT DESCRIBE ON KEYSPACE keyspace_name TO ROLE role_name;

Tip: See and .

Default: false

role_manager

The DSE Role Manager supports LDAP roles and internal roles supported by the CassandraRoleManager. Role options are stored in the dse_security keyspace. When using the DSE Role Manager, increase the replication factor of the dse_security keyspace. Role managers other than DseRoleManager are deprecated and not supported. Some security features might not work correctly if other role managers are used.

Important: Use only role manager implementations bundled with DSE.

Default: com.datastax.bdp.cassandra.auth.DseRoleManager

roles_validity_in_ms

Validity period for roles cache in milliseconds. Determines how long to cache the list of roles assigned to the user; users may have several roles, either through direct assignment or inheritance (a role that has been granted to another role). Adjust this setting based on the complexity of your role hierarchy, tolerance for role changes, the number of nodes in your environment, and activity level of the cluster.

Fetching permissions can be an expensive operation, so this setting allows flexibility. Granted roles are cached for authenticated sessions in AuthenticatedUser. After the specified time elapses, role validity is rechecked. Disabled automatically when internal authentication is not enabled when using DseAuthenticator.

• 0 - disable role caching
• milliseconds - how long to cache the list of roles assigned to the user

Default: 120000 (2 minutes)

roles_update_interval_in_ms

Refresh interval for roles cache. After this interval, cache entries become eligible for refresh. On next access, the database schedules an async reload, and returns the old value until the reload completes. If roles_validity_in_ms is non-zero, then this value must also be non-zero. When not set, the default is the same value as roles_validity_in_ms.

Default: commented out (120000) 

permissions_validity_in_ms

How long permissions in cache remain valid to manage performance impact of permissions queries. Fetching permissions can be resource intensive. Set the cache validity period to your security tolerances. The cache is used for the standard authentication and the row-level access control (RLAC) cache. The cache is quite effective at small durations.

• 0 - disable permissions cache
• milliseconds - time, in milliseconds

CAUTION: does not automatically invalidate cached permissions. Permissions are invalidated the next time they are refreshed.

Default: 120000 (2 minutes)

permissions_update_interval_in_ms

Sets refresh interval for the standard authentication cache and the row-level access control (RLAC) cache. After this interval, cache entries become eligible for refresh. On next access, the database schedules an async reload and returns the old value until the reload completes. If permissions_validity_in_ms is non-zero, the value for roles_update_interval_in_ms must also be non-zero. When not set, the default is the same value as permissions_validity_in_ms.

Default: commented out (2000) 

permissions_cache_max_entries

The maximum number of entries that are held by the standard authentication cache and row-level access control (RLAC) cache. With the default value of 1000, the RLAC permissions cache can have up to 1000 entries in it, and the standard authentication cache can have up to 1000 entries. This single option applies to both caches. To size the permissions cache for use with , use this formula:

numRlacUsers * numRlacTables + 100

If this option is not present in cassandra.yaml, manually enter it to use a value other than 1000. See .

Default: not set (1000) 

Inter-node encryption options

Node-to-node (internode) encryption protects data that is transferred between nodes in a cluster using SSL.

server_encryption_options:
    internode_encryption: none
    keystore: resources/dse/conf/.keystore
    keystore_password: cassandra
    truststore: resources/dse/conf/.truststore
    truststore_password: cassandra
    # More advanced defaults below:
    # protocol: TLS
    # algorithm: SunX509
    # store_type: JKS
    # cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
    # require_client_auth: false
    # require_endpoint_verification: false

server_encryption_options

Inter-node encryption options. If enabled, you must also generate keys and provide the appropriate key and truststore locations and passwords. No custom encryption options are supported.

Tip: The passwords used in these options must match the passwords used when generating the keystore and truststore. For instructions on generating these files, see Creating a Keystore to Use with JSSE.

Tip: See .

internode_encryption

Encryption options for of inter-node communication using the TLS_RSA_WITH_AES_128_CBC_SHA cipher suite for authentication, key exchange, and encryption of data transfers. Use the DHE/ECDHE ciphers, such as TLS_DHE_RSA_WITH_AES_128_CBC_SHA if running in (Federal Information Processing Standard) FIPS 140 compliant mode.

• all - Encrypt all inter-node communications
• none - No encryption
• dc - Encrypt the traffic between the datacenters
• rack - Encrypt the traffic between the racks

Default: none

keystore

Relative path from DSE installation directory or absolute path to the Java keystore (JKS) suitable for use with Java Secure Socket Extension (JSSE), which is the Java version of the Secure Sockets Layer (SSL), and Transport Layer Security (TLS) protocols. The keystore contains the private key used to encrypt outgoing messages.

Default: resources/dse/conf/.keystore

keystore_password

Password for the keystore. This must match the password used when generating the keystore and truststore.

Default: cassandra

truststore

Relative path from DSE installation directory or absolute path to truststore containing the trusted certificate for authenticating remote servers.

Default: resources/dse/conf/.truststore

truststore_password

Password for the truststore.

Default: cassandra

protocol

Default: commented out (TLS)

algorithm

Default: commented out (SunX509)

store_type

Valid types are JKS, JCEKS, and PKCS12.

Note: PKCS11 is not supported.

Default: commented out (JKS)

truststore_type

Valid types are JKS, JCEKS, and PKCS12.

Attention: PKCS11 is not supported. Also, due to an OpenSSL issue, you cannot use a PKCS12 truststore that was generated via OpenSSL. For example, a truststore generated via the following command will not work with DSE:

openssl pkcs12 -export -nokeys -out truststore.pfx -in intermediate.chain.pem

However, truststores generated via Java's keytool and then converted to PKCS12 work with DSE. Example:

keytool -importcert -alias rootca -file rootca.pem -keystore truststore.jks

keytool -importcert -alias intermediate -file intermediate.pem -keystore truststore.jks

keytool -importkeystore -srckeystore truststore.jks -destkeystore truststore.pfx -deststoretype pkcs12 

Default: commented out (JKS)

cipher_suites

Supported ciphers:

• TLS_RSA_WITH_AES_128_CBC_SHA
• TLS_RSA_WITH_AES_256_CBC_SHA
• TLS_DHE_RSA_WITH_AES_128_CBC_SHA
• TLS_DHE_RSA_WITH_AES_256_CBC_SHA
• TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
• TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA

Default: commented out

require_client_auth

Whether to enable certificate authentication for node-to-node (internode) encryption. When not set, the default is false.

Default: commented out (false)

require_endpoint_verification

Whether to verify the connected host and the host IP address in the certificate match. If set to true, the endpoint that you specify when generating the certificate key must be an IP address. Do not specify a DNS hostname. Example with a correctly specified IP address:

keytool -genkeypair -keyalg RSA \
    -alias node0 \
    -keystore my_keystore.jks \
    -storepass cassandra \
    -keypass cassandra \
    -validity 730 \
    -keysize 2048 \
    -dname "CN=node0, OU=lacerda-ssl, O=Datastax, C=CC" \
    -ext "san=ip:10.101.35.236"

Default: false

Client-to-node encryption options

client_encryption_options:
    enabled: false
    # If enabled and optional is set to true, encrypted and unencrypted connections over native transport are handled.
    optional: false
    keystore: resources/dse/conf/.keystore
    keystore_password: cassandra
    # require_client_auth: false
    # Set trustore and truststore_password if require_client_auth is true
    # truststore: resources/dse/conf/.truststore
    # truststore_password: cassandra
    # More advanced defaults below:
    # protocol: TLS
    # algorithm: SunX509
    # store_type: JKS
    # cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]

client_encryption_options

Whether to enable client-to-node encryption. You must also generate keys and provide the appropriate key and truststore locations and passwords. There are no custom encryption options enabled for DataStax Enterprise.

Advanced settings:

enabled

Whether to enable client-to-node encryption.

Default: false

optional

When optional is selected, both encrypted and unencrypted connections over native transport are allowed. That is a necessary transition state to facilitate enabling client to node encryption on live clusters without inducing an outage for existing unencrypted clients. Typically, once existing clients are migrated to encrypted connections, optional is unselected in order to enforce native transport encryption.

Default: false

keystore

Relative path from DSE installation directory or absolute path to the Java keystore (JKS) suitable for use with Java Secure Socket Extension (JSSE), which is the Java version of the Secure Sockets Layer (SSL), and Transport Layer Security (TLS) protocols. The keystore contains the private key used to encrypt outgoing messages.

Default: resources/dse/conf/.keystore

keystore_password

Password for the keystore.

Default: cassandra

require_client_auth

Whether to enable certificate authentication for client-to-node encryption. When not set, the default is false.

Note: When set to true, client certificates must be present on all nodes in the cluster.

Default: commented out (false)

truststore

Relative path from DSE installation directory or absolute path to truststore containing the trusted certificate for authenticating remote servers.

Default: resources/dse/conf/.truststore

truststore_password

Password for the truststore. This must match the password used when generating the keystore and truststore.

Note: Truststore password and path is only required when require_client_auth is set to true.

Default: cassandra

protocol

Default: commented out (TLS)

algorithm

Default: commented out (SunX509)

store_type

Valid types are JKS, JCEKS and PKCS12. For file-based keystores, use PKCS12.

Note: PKCS11 is not supported.

Default: commented out (JKS)

truststore_type

Valid types are JKS, JCEKS, and PKCS12.

Attention: PKCS11 is not supported. Also, due to an OpenSSL issue, you cannot use a PKCS12 truststore that was generated via OpenSSL. For example, a truststore generated via the following command will not work with DSE:

openssl pkcs12 -export -nokeys -out truststore.pfx -in intermediate.chain.pem

However, truststores generated via Java's keytool and then converted to PKCS12 work with DSE. Example:

keytool -importcert -alias rootca -file rootca.pem -keystore truststore.jks

keytool -importcert -alias intermediate -file intermediate.pem -keystore truststore.jks

keytool -importkeystore -srckeystore truststore.jks -destkeystore truststore.pfx -deststoretype pkcs12 

Default: commented out (JKS)

cipher_suites

Supported ciphers:

• TLS_RSA_WITH_AES_128_CBC_SHA
• TLS_RSA_WITH_AES_256_CBC_SHA
• TLS_DHE_RSA_WITH_AES_128_CBC_SHA
• TLS_DHE_RSA_WITH_AES_256_CBC_SHA
• TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
• TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA

Default: commented out

Transparent data encryption options

transparent_data_encryption_options:
    enabled: false
    chunk_length_kb: 64
    cipher: AES/CBC/PKCS5Padding
    key_alias: testing:1
    # CBC IV length for AES must be 16 bytes, the default size
    # iv_length: 16
    key_provider:
      - class_name: org.apache.cassandra.security.JKSKeyProvider
        parameters:
          - keystore: conf/.keystore
            keystore_password: cassandra
            store_type: JCEKS
            key_password: cassandra

transparent_data_encryption_options

DataStax Enterprise supports this option only for backward compatibility. When using DSE, configure data encryption options in the dse.yaml; see .

TDE properties:

• enabled: (Default: false)
• chunk_length_kb: (Default: 64)
• cipher: options:
  • AES
  • CBC
  • PKCS5Padding
• key_alias: testing:1
• iv_length: 16
  Note: iv_length is commented out in the default cassandra.yaml file. Uncomment only if cipher is set to AES. The value must be 16 (bytes).
• key_provider:
  • class_name: org.apache.cassandra.security.JKSKeyProvider

    parameters:

    • keystore: conf/.keystore
    • keystore_password: cassandra
    • store_type: JCEKS
    • key_password: cassandra

SSL Ports

ssl_storage_port: 7001
native_transport_port_ssl: 9142

ssl_storage_port

The SSL port for encrypted communication. Unused unless enabled in encryption_options. Follow security best practices, do not expose this port to the internet. Apply firewall rules.

Default: 7001

native_transport_port_ssl

Dedicated SSL port where the CQL native transport listens for clients with encrypted communication. For security reasons, do not expose this port to the internet. Firewall it if needed.

• commented out (disabled) - the native_transport_port will encrypt all traffic
• port number different than native_transport_port - use encryption for native_transport_port_ssl, keep native_transport_port unencrypted to use both unencrypted and encrypted traffic

Default: 9142

Continuous paging options

continuous_paging:
    max_concurrent_sessions: 60
    max_session_pages: 4
    max_page_size_mb: 8
    max_local_query_time_ms: 5000
    client_timeout_sec: 600
    cancel_timeout_sec: 5
    paused_check_interval_ms: 1

continuous_paging

Options to tune continuous paging that pushes pages, when requested, continuously to the client:

• Maximum memory used:

  max_concurrent_sessions ⨉ max_session_pages ⨉ max_page_size_mb

  Default: calculated (60 ⨉ 4 ⨉ 8 = 1920 MB)

Guidance

• Because memtables and SSTables are used by the continuous paging query, you can define the maximum period of time during which memtables cannot be flushed and compacted SSTables cannot be deleted.
• If fewer threads exist than sessions, a session cannot execute until another one is swapped out.
• Distributed queries (CL > ONE or non-local data) are swapped out after every page, while local queries at CL = ONE are swapped out after max_local_query_time_ms.

max_concurrent_sessions

The maximum number of concurrent sessions. Additional sessions are rejected with an unavailable error.

Default: 60

max_session_pages

The maximum number of pages that can be buffered for each session. If the client is not reading from the socket, the producer thread is blocked after it has prepared max_session_pages.

Default: 4

max_page_size_mb

The maximum size of a page, in MB. If an individual CQL row is larger than this value, the page can be larger than this value.

Default: 8

max_local_query_time_ms

The maximum time for a local continuous query to run. When this threshold is exceeded, the session is swapped out and rescheduled. Swapping and rescheduling ensures the release of resources that prevent the memtables from flushing and ensures fairness when max_threads < max_concurrent_sessions. Adjust when high write workloads exist on tables that have continuous paging requests.

Default: 5000

client_timeout_sec

How long the server will wait, in seconds, for clients to request more pages if the client is not reading and the server queue is full.

Default: 600

cancel_timeout_sec

How long to wait before checking if a paused session can be resumed. Continuous paging sessions are paused because of backpressure or when the client has not request more pages with backpressure updates.

Default: 5

paused_check_interval_ms

How long to wait, in milliseconds, before checking if a continuous paging sessions can be resumed, when that session is paused because of backpressure.

Default: 1

Fault detection setting

# phi_convict_threshold: 8

phi_convict_threshold

The sensitivity of the failure detector on an exponential scale. Generally, this setting does not need adjusting.

Tip: See .

When not set, the internal value is 8.

Default: commented out (8)

Memory leak detection settings

#leaks_detection_params:
#  sampling_probability: 0
#  max_stacks_cache_size_mb: 32
#  num_access_records: 0
#  max_stack_depth: 30

sampling_probability

The sampling probability to track for the specified resource. For resources tracked, see nodetool leaksdetection.

• 0 - disable tracking. Default.
• 1 - enable tracking all the time
• A number between 0 and 1 - the percentage of time to randomly track a resource. For example, 0.5 will track resources 50% of the time.

CAUTION: Tracking incurs a significant stack trace collection cost for every access and consumes heap space. Enable tracking only when directed by DataStax Support.

Default: commented out (0)

max_stacks_cache_size_mb

Set the size of the cache for call stack traces. Stack traces are used to debug leaked resources, and use heap memory. Set the amount of heap memory dedicated to each resource by setting the max stacks cache size in MB.

Default: commented out (32)

num_access_records

Set the average number of stack traces kept when a resource is accessed. Currently only supported for chunks in the cache.

Default: commented out (0)

max_stack_depth

Set the depth of the stack traces collected. Changes only the depth of the stack traces that will be collected from the time the parameter is set. Deeper stacks are more unique, so increasing the depth may require increasing stacks_cache_size_mb.

Default: commented out (30)