No DataStax Enterprise processing but high CPU usage

Extremely high CPU usage but no DataStax Enterprise processing on Linux platforms.

The DataStax Enterprise Help Center also provides troubleshooting information.

Extremely high CPU usage but no DataStax Enterprise processing on Linux platforms.

Check the CPU usage for the process khugepaged. It may run as high as 100%, blocking other processes.

Cause:

Many modern Linux distributions ship with Transparent Hugepages enabled by default. When Linux uses Transparent Hugepages, the kernel tries to allocate memory in large chunks (usually 2 MB), rather than 4K. This can improve performance by reducing the number of pages the CPU must track. However, some applications still allocate memory based on 4K pages. This can cause noticeable performance problems when Linux tries to defrag 2 MB pages. For more information, see Cassandra Java Huge Pages and this RedHat bug report.

Possible solutions:
  • A temporary fix: drop caches by entering:
    sync && echo 3 > /proc/sys/vm/drop_caches
  • A better solution: disable defrag for hugepages by entering:
    echo never | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
  • Another alternative: add -XX:+AlwaysPreTouch to the jvm.options file. This change should be tested carefully before being put into production. For details, see Tuning Java Virtual Machine and blog post.