Changing the Hadoop log directory

Add the HADOOP_LOG_DIR environment variable to the dse-env.sh file to recognize changes to the default log directory used by the Hadoop component that is integrated into DataStax Enterprise.

DataStax Enterprise does not recognize changes to the default log directory used by the Hadoop component integrated into DataStax Enterprise 4.0 unless you add the HADOOP_LOG_DIR environment variable to the dse-env.sh file, as described in this topic.

The Hadoop environment variables are set in the hadoop-env.sh file. In this file, the HADOOP_LOG_DIR default configuration is:

  • Tarball installs: /var/log/hadoop
  • Packaged installs: /etc/dse/hadoop/hadoop-env.sh

If you change the default Hadoop log directory environment variable in hadoop-env.sh and restart DataStax Enterprise, the change will not be recognized.

The workaround, if you want to change the default Hadoop log directory, is to add HADOOP_LOG_DIR to the following file:

  • Tarball installs: install_dir/bin/dse-env.sh
  • Packaged installs: /etc/dse/dse-env.sh

Procedure

  1. In the dse-env.sh file, you will see comments about exactly where to add the command to configure the environment variable. For example:
    #!/bin/sh
    
    # Add any environment overrides you need here. This is where users
    # may set third-party variables such as HADOOP_LOG_DIR
    
    export HADOOP_LOG_DIR=/var/log/hadoop/new_log_location
    
    # ==================================
    # don't change after this.
    if [ -r "`dirname "$0"`/dse.in.sh" ]; then
    . . .
  2. Restart DataStax Enterprise after configuring the new log location.

    In a packaged installation, DataStax Enterprise loads the environment variable change using /usr/share/dse/dse.in.sh after you restart the node.