Insufficient user resource limits errors
Insufficient resource limits may result in a number of errors in Cassandra and OpsCenter.
Insufficient resource limits may result in a number of errors in Cassandra and OpsCenter.
Cassandra errors
- Insufficient as (address space) or memlock setting
-
ERROR [SSTableBatchOpen:1 ] 2012-07-25 15:46:02,913 AbstractCassandraDaemon.java (line 139 ) Fatal exception in thread Thread [SSTableBatchOpen:1,5,main ] java.io.IOError: java.io.IOException: Map failed at ...
- Insufficient memlock settings
-
WARN [main ] 2011-06-15 09:58:56,861 CLibrary.java (line 118 ) Unable to lock JVM memory (ENOMEM ). This can result in part of the JVM being swapped out, especially with mmapped I/O enabled. Increase RLIMIT_MEMLOCK or run Cassandra as root.
- Insufficient nofiles setting
-
WARN 05:13:43,644 Transport error occurred during acceptance of message. org.apache.thrift.transport.TTransportException: java.net.SocketException: Too many open files ...
- Insufficient nofiles setting
-
ERROR [MutationStage:11 ] 2012-04-30 09:46:08,102 AbstractCassandraDaemon.java (line 139 ) Fatal exception in thread Thread [MutationStage:11,5,main ] java.lang.OutOfMemoryError: unable to create new native thread
Recommended settings
You can view the current limits using the ulimit -a command. Although limits can also be temporarily set using this command, DataStax recommends making the changes permanent:
Packaged installs:
Ensure that the following
settings are included in the
/etc/security/limits.d/cassandra.conf
file:
cassandra - memlock unlimited cassandra - nofile 100000 cassandra - nproc 32768 cassandra - as unlimited
Tarball installs:
Ensure that the following settings
are included in the
/etc/security/limits.conf
file:
* - memlock unlimited * - nofile 100000 * - nproc 32768 * - as unlimited
If you run Cassandra as root, some Linux distributions such as
Ubuntu, require setting the limits for root explicitly instead of using *:
root - memlock unlimited root - nofile 100000 root - nproc 32768 root - as unlimited
For CentOS, RHEL, OEL systems, also set the
nproc
limits in
/etc/security/limits.d/90-nproc.conf
:
* - nproc 32768
For all installations, add the following line to
/etc/sysctl.conf
:
vm.max_map_count = 1048575
To make the changes take effect, reboot the server or run the
following command:
$ sudo sysctl -p
To confirm the limits are applied to the Cassandra process, run the
following command where
<pid>
is the process ID of the currently
running Cassandra process:
$ cat /proc/<pid>/limits
OpsCenter errors
See the OpsCenter Troubleshooting documentation.