Set kernel parameters
Configure the following kernel parameters for optimal traffic and user limits.
Run the following command to view all current Linux kernel settings:
sudo sysctl -aTCP settings
During low traffic intervals, a firewall configured with an idle connection timeout can close connections to local nodes and nodes in other data centers. To prevent connections between nodes from timing out, set the following network kernel settings:
-
Set the following TCP keepalive timeout values:
sudo sysctl -w \ net.ipv4.tcp_keepalive_time=60 \ net.ipv4.tcp_keepalive_probes=3 \ net.ipv4.tcp_keepalive_intvl=10These values set the TCP keepalive timeout to 60 seconds with 3 probes, 10 seconds gap between each. The settings detect dead TCP connections after 90 seconds (60 + 10 + 10 + 10). When the additional traffic is negligible, it is safe to persist these TCP keepalive timeout settings.
-
Change the following settings to handle thousands of concurrent connections used by the database:
sudo sysctl -w \ net.core.rmem_max=16777216 \ net.core.wmem_max=16777216 \ net.core.rmem_default=16777216 \ net.core.wmem_default=16777216 \ net.core.optmem_max=40960 \ net.ipv4.tcp_rmem='4096 87380 16777216' \ net.ipv4.tcp_wmem='4096 65536 16777216'
|
Instead of changing the system TCP settings, you can prevent reset connections during streaming by tuning the streaming_keep_alive_period_in_secs setting in |
Set user resource limits
Use the ulimit -a command to view the current limits. Although limits can also be temporarily set using this command, DataStax recommends making the changes permanent.
For more information, see Recommended production settings.
Debian-based systems
-
Edit the
/etc/pam.d/sufile and uncomment the following line to enable thepam_limits.somodule:session required pam_limits.soThis change to the PAM configuration file ensures that the system reads the files in the
/etc/security/limits.ddirectory. -
If you run DSE as root, some Linux distributions (such as Ubuntu), require setting the limits for the root user explicitly instead of using <cassandra_user>:
root - memlock unlimited root - nofile 1048576 root - nproc 32768 root - as unlimited
RHEL-based systems
-
Set the nproc limits to 32768 in the
/etc/security/limits.d/90-nproc.confconfiguration file:cassandra_user <- nproc 32768>
All systems
-
Add the following line to
/etc/sysctl.conf:vm.max_map_count = 1048575 -
Open the configuration file for your installation type:
Installation type Configuration file Tarball installation
/etc/security/limits.confPackage installation
/etc/security/limits.d/cassandra.conf -
Configure the following settings for the
<cassandra_user>in the configuration file:<cassandra_user> - memlock unlimited <cassandra_user> - nofile 1048576 <cassandra_user> - nproc 32768 <cassandra_user> - as unlimited -
Reboot the server or run the following command to make all changes take effect:
sudo sysctl -p
Persist updated settings
-
Add the following values to the /etc/sysctl.conf file:
net.ipv4.tcp_keepalive_time=60 net.ipv4.tcp_keepalive_probes=3 net.ipv4.tcp_keepalive_intvl=10 net.core.rmem_max=16777216 net.core.wmem_max=16777216 net.core.rmem_default=16777216 net.core.wmem_default=16777216 net.core.optmem_max=40960 net.ipv4.tcp_rmem=4096 87380 16777216 net.ipv4.tcp_wmem=4096 65536 16777216
-
Load the settings using one of the following commands:
sudo sysctl -p /etc/sysctl.confsudo sysctl -p /etc/sysctl.d/*.conf -
To confirm the user limits are applied to the DSE process, run the following command where <pid> is the process ID of the currently running DSE process:
cat /proc/<pid>/limits
