DataStax Enterprise recommended settings for Docker
Follow the recommended guidance and settings for using DataStax Enterprise (DSE) with Docker.
To ensure your success when using Docker, follow the recommended guidance and settings for using DataStax Enterprise (DSE) with Docker.
General guidance
Software versions
The official DataStax images include the latest DataStax Agent version at the time of official image build. If you require a version of the DataStax Agent that differs from the one included with the official image, you must build an image that includes the required versions.
Hardware settings
The optimum readahead
setting for RAID on
SSDs (in Amazon EC2) is 8 KB, the same as it is for non-RAID SSDs. For details,
see Optimizing SSDs.
Typically, a readahead
of 128 is recommended.
Check to ensure setra
is not set to 65536:
sudo blockdev --report /dev/spinning_disk
To set setra:
sudo blockdev --setra 128 /dev/spinning_disk
System settings
Because time is not namespaced in the Linux kernel, containers share the clock with the Docker host machine. Ensure that clocks are synchronized on the host machines and containers by configuring NTP or other methods on the host machines.
Swapping must be disabled for performance and node stability. Run the following command on the Docker host to disable swap. The Docker host passes this setting to the container.
See Disabling swap for DSE 6.7 | DSE 6.0 | DSE 5.1 | DDAC.
sudo swapoff --all
- To disable swap per container, see Preventing a container from using SWAP in the Docker documentation.
- To make this change permanent, remove all swap file entries from /etc/fstab.
To ensure optimal performance, do not use governors that lower the CPU frequency. Instead,
reconfigure all CPUs to use the performance
governor on the Docker hosts.
See Disable CPU frequency scaling for DSE 6.7 | DSE 6.0 | DSE 5.1 | DDAC.
for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
do
[ -f $CPUFREQ ] || continue
echo -n performance > $CPUFREQ
done
THP can cause performance issues in DSE or DDAC when it defragments 4 K chunks into 2 MB
chunks. To disable defrag
, run the following command on the Docker host:
echo never | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
See Check Java Hugepages settings for DSE 6.7 | DSE 6.0 | DSE 5.1 | DDAC.
All containers by default inherit user limits from the Docker daemon. In production
environments, DSE expects the following changes to ulimit
:
ulimit -n 100000 # nofile: max number of open files
ulimit -l unlimited # memlock: maximum locked-in-memory address space
- Run the following command to check the Docker daemon defaults for
ulimits
:docker run --rm ubuntu /bin/bash -c 'ulimit -a'
- To set
ulimit
for Docker containers, run thedocker run
command with the followingulimit
options:--ulimit nofile=100000:100000 --ulimit nproc=32768 --ulimit memlock=-1:-1
DSE tries to lock memory using mlock
. When running in Docker, that capability is disabled. To enable
mlock
, add the following option to the docker run
command:
--cap-add=IPC_LOCK
On the Docker host, check the value of vm.max_map_count
, which should be set
to 1048575.
cat /proc/sys/vm/max_map_count
To set the value of vm.max_map_count
, add the following line to
/etc/sysctl.conf, and then run sysctl -p
to propagate the
changes.
vi /etc/sysctl.conf
vm.max_map_count = 1048575
sudo sysctl -p
See Set user resource limits for DSE 6.7 | DSE 6.0 | DSE 5.1 | DDAC.
For each container in production environments, explicitly set the JVM heap size using the
JVM_EXTRA_OPTS
environment variable with the docker run
command.
For example, to use 16 GB for the JVM heap, run the docker run
command with
the following option:
docker run -e JVM_EXTRA_OPTS="-Xms16g -Xmx16g"
Storage and resource requirements
For advanced configuration management, DataStax provides a mechanism for modifying configurations without replacing or customizing DataStax Docker containers. When any of the approved configuration files are mounted to a host volume, the files are mapped automatically within the container. See Using the DSE configuration volume.
The DSE Docker container writes all node-specific data in the directories under
/var/lib/cassandra/ by default. To persist this data, map the data
directories inside the container to a directory on the host file system using the
-v option with the docker run
command, or by using a volume driver.
docker run
command with the following
option:docker run -v /dse/data:/var/lib/cassandra
Hosting the /var/lib/cassandra directory outside the container with the -v option allows the container to be deleted and recreated without losing data. See Persisting data.
If using the Docker devicemapper
storage driver, do not use the default
loop-lvm
mode, which is only appropriate for testing. Instead, configure
docker-engine
to use direct-lvm mode, which is suitable for production
environments.
See the DataStax Developer Blog Running DSE on Microsoft Windows Using Docker. When running Docker for Windows, the default resources allocated to the Linux VM running docker are 2 GB RAM and 2 CPUs. Adjust these resources as appropriate to meet the requirements for your containers. See Getting Started in Docker Desktop for Windows.
Network considerations
Because the default network settings in Docker (via Linux bridge) slows networking
considerably, do not use these network settings in production environments. Instead, use docker host
networking by adding the --network host
option to the docker
run
command, or use a plugin that can manage IP ranges across clusters of hosts. The
host networking limits the number of nodes per Docker host to one, which is the recommended
configuration to use in production.
docker run -d --network host --name container_name