Set kernel parameters

Configure the kernel for optimal traffic and user limits.

Configure the following kernel parameters for optimal traffic and user limits.

TCP settings

During low traffic intervals, a firewall configured with an idle connection timeout can close connections to local nodes and nodes in other data centers. To prevent connections between nodes from timing out, set the following network kernel settings:

  1. Set the following TCP keepalive timeout values:
    sudo sysctl -w \
    net.ipv4.tcp_keepalive_time=60 \
    net.ipv4.tcp_keepalive_probes=3 \
    net.ipv4.tcp_keepalive_intvl=10

    These values set the TCP keepalive timeout to 60 seconds with 3 probes, 10 seconds gap between each. The settings detect dead TCP connections after 90 seconds (60 + 10 + 10 + 10). The additional traffic is negligible, and permanently leaving these settings is not an issue. See Firewall idle connection timeout causing nodes to lose communication during low traffic times on Linux .

  2. Change the following settings to handle thousands of concurrent connections used by the database:
    sudo sysctl -w \
    net.core.rmem_max=16777216 \
    net.core.wmem_max=16777216 \
    net.core.rmem_default=16777216 \
    net.core.wmem_default=16777216 \
    net.core.optmem_max=40960 \
    net.ipv4.tcp_rmem=4096 87380 16777216 \
    net.ipv4.tcp_wmem=4096 65536 16777216

Set user resource limits

Use the ulimit -a command to view the current limits. Although limits can also be temporarily set using this command, DataStax recommends making the changes permanent.

For more information, see Insufficient user resource limits errors.

Debian-based systems
  1. Edit the /etc/pam.d/su file and uncomment the following line to enable the pam_limits.so module:
    session    required   pam_limits.so

    This change to the PAM configuration file ensures that the system reads the files in the /etc/security/limits.d directory.

  2. If you run DSE as root, some Linux distributions (such as Ubuntu), require setting the limits for the root user explicitly instead of using cassandra_user:
    root - memlock unlimited
    root - nofile 1048576
    root - nproc 32768
    root - as unlimited
RHEL-based systems
  1. Set the nproc limits to 32768 in the /etc/security/limits.d/90-nproc.conf configuration file:
    cassandra_user - nproc 32768
All systems
  1. Add the following line to /etc/sysctl.conf:
    vm.max_map_count = 1048575
  2. Open the configuration file for your installation type:
    Installation type Configuration file
    Tarball installation /etc/security/limits.conf
    Package installation /etc/security/limits.d/cassandra.conf
  3. Configure the following settings for the <cassandra_user> in the configuration file:
    <cassandra_user> - memlock unlimited
    <cassandra_user> - nofile 1048576
    <cassandra_user> - nproc 32768
    <cassandra_user> - as unlimited
  4. Reboot the server or run the following command to make all changes take effect:
    sudo sysctl -p

Persist updated settings

  1. Add the following values to the /etc/sysctl.conf file:
    net.ipv4.tcp_keepalive_time=60
    net.ipv4.tcp_keepalive_probes=3
    net.ipv4.tcp_keepalive_intvl=10
    net.core.rmem_max=16777216
    net.core.wmem_max=16777216
    net.core.rmem_default=16777216
    net.core.wmem_default=16777216
    net.core.optmem_max=40960
    net.ipv4.tcp_rmem=4096 87380 16777216
    net.ipv4.tcp_wmem=4096 65536 16777216
  2. Load the settings using one of the following commands:
    sudo sysctl -p /etc/sysctl.conf
    sudo sysctl -p /etc/sysctl.d/*.conf
  3. To confirm the user limits are applied to the DSE process, run the following command where pid is the process ID of the currently running DSE process:
    cat /proc/pid/limits