Recommended production settings
Recommendations for production environments.
Recommendations for production environments; adjust them accordingly for your implementation.
Disable zone_reclaim_mode on NUMA systems¶The Linux kernel can be inconsistent in enabling/disabling zone_reclaim_mode. This can result in odd performance problems.
echo 0 > /proc/sys/vm/zone_reclaim_mode
For more information, see Peculiar Linux kernel performance problem on NUMA systems.
User resource limits¶
You can view the current limits using the ulimit -a command. Although limits can also be temporarily set using this command, DataStax recommends making the changes permanent: ¶
cassandra - memlock unlimited cassandra - nofile 100000 cassandra - nproc 32768 cassandra - as unlimited
* - memlock unlimited * - nofile 100000 * - nproc 32768 * - as unlimited
root - memlock unlimited root - nofile 100000 root - nproc 32768 root - as unlimited
* - nproc 32768
vm.max_map_count = 131072
$ sudo sysctl -p
$ cat /proc/<pid>/limits
For more information, see Insufficient user resource limits errors.
You must disable swap entirely. Failure to do so can severely lower performance. Because Cassandra has multiple replicas and transparent failover, it is preferable for a replica to be killed immediately when memory is low rather than go into swap. This allows traffic to be immediately redirected to a functioning replica instead of continuing to hit the replica that has high latency due to swapping. If your system has a lot of DRAM, swapping still lowers performance significantly because the OS swaps out executable code so that more DRAM is available for caching disks.
If you insist on using swap, you can set vm.swappiness=1. This allows the kernel swap out the absolute least used parts.
$ sudo swapoff --all
To make this change permanent, remove all swap file entries from /etc/fstab.
For more information, see Nodes seem to freeze after some period of time.
The clocks on all nodes should be synchronized. You can use NTP (Network Time Protocol) or other methods.
This is required because columns are only overwritten if the timestamp in the new version of the column is more recent than the existing column.
Optimum blockdev --setra settings for RAID¶
Typically, a readahead of 128 is recommended, especially on Amazon EC2 RAID0 devices.
Check to ensure setra is not set to 65536:
sudo blockdev --report /dev/<device>
To set setra:
sudo blockdev --setra 128 /dev/<device>
Java Virtual Machine¶
Java Native Access¶
Java Native Access (JNA) is required for production installations.