Add nodes to DSE Multi-Instance

This feature is deprecated in DSE 6.8.

With package installs, the dse add-node command simplifies adding and configuring DataStax Enterprise (DSE) nodes on a host machine.

Tarball installs do not support adding more nodes on a single host machine. To install DSE Multi-Instance in a tarball installation, unpack the tarball in multiple locations on a single host machine. Each tarball installation becomes a DSE node on the host machine.

On the host machine, the DSE Multi-Instance root directory is /etc/defaults. This default location is not configurable. The node type is defined in the /etc/defaults/<dse-nodeId> file.

These actions occur for each node that is added:

  • The node configuration is modified according to the command arguments.

  • A script is created so that the node can be started and stopped.

  • The run levels are updated to the default values so that the node is started and stopped when the host machine is booted or halted.

  • The /etc/default/<dse-nodeId> file is created to set the default node type as a transactional node.

  • With DSE Multi-Instance, when you run the dse command on a node in the host machine, the node configuration is read from:

    • Package installations: /etc/dse/serverconfig/<dse-nodeId>

    • Tarball installations: the /etc/dse directory is the default configuration location in each location where you installed DSE.

Configuration files for DSE Multi-Instance

With DSE Multi-Instance, multiple DSE nodes reside on a single host machine. To segregate the configuration for each DSE node, configuration and operational files are stored in node-specific directory structures.

For example, in addition to /etc/dse/dse.yaml, additional dse.yaml files for each DSE Multi-Instance node are stored in sibling directories named after each node’s ID, such as /etc/dse-nodeId/dse.yaml. In each node’s dse.yaml file (at /etc/dse-nodeId/dse.yaml), the generated server_id value uniquely identifies the physical server on which multiple instances are running. This value is unique for each database instance.

Example of primary and sibling directories for DSE Multi-Instance nodes
Path Description

/etc/dse/dse.yaml

Your DSE installation’s primary dse.yaml configuration file.

/etc/dse-node1/dse.yaml

The dse.yaml configuration file for a Multi-Instance node with the ID node1.

/etc/dse-node2/dse.yaml

The dse.yaml configuration file for a Multi-Instance node with the ID node2.

Two files control the initial configuration of a DSE Multi-Instance node:

Configuration files to initialize a Multi-Instance node
Path Description

/etc/dse/serverconfig/dse-nodeId

Specifies the directories for the node’s configuration files.

/etc/default/dse-nodeId

Defines essential start up behavior for the node, including the node type (transactional, DSE Search, DSE Analytics, or DSE Graph) and the number of retries for the DSE service to start.

Once a node is initialized, its configuration and operational files are stored in generated, node-specific directories, as described in the following table. These directories are created on the host machine for each node.

Default generated directories for DSE Multi-Instance nodes
Path Description

/etc/defaults

Root directory for each DSE Multi-Instance node. This location is not configurable.

Contains the /etc/defaults/dse-nodeId file that sets the node type and other essential start up behavior.

/etc/dse-nodeId/dse.yaml

The dse.yaml configuration file for each node.

/etc/dse-nodeId/cassandra/cassandra.yaml

The cassandra.yaml configuration file for each node.

/var/lib/dse-nodeId/data

Root database data directory on each node.

Define with dse add-node --data-directory=directory …​

/var/log/dse-nodeId/cassandra

Default database log file directory for each node. Contains files like audit.log, output.log, solrvalidation.log, and system.log log.

Define with dse add-node --logs-directory=directory …​

/var/run/dse-nodeId

Database process ID (pid) directory for each node.

/var/lib/dse-nodeId/saved_caches

Table key and row caches for each node.

Define with dse add-node --saved-caches-directory=directory …​

/var/lib/dse-nodeId/commitlog

Commitlog files directory for each node.

Define with dse add-node --commit-directory=directory …​

/var/lib/dse-nodeId/hints

Hints directory for each node.

Define with dse add-node --hints-directory=directory …​

/etc/dse-nodeId/spark/spark-env.sh

Spark configuration file for each node.

/var/lib/dse-nodeId/spark/worker

Spark worker data directory for each node.

Define with dse add-node --spark-worker-directory=directory …​

/var/lib/dse-nodeId/spark/rdd

Local Spark worker data directory for each node.

Define with dse add-node --spark-local-directory=directory …​

/var/log/dse-nodeId/spark

Spark logs for each node.

Define with dse add-node --spark-log-directory=directory …​

/etc/dse-nodeId/cassandra/logback.xml

The logback configuration file for each node.

/etc/dse-nodeId/solr

Solr configuration files for each node.

/var/log/dse-nodeId/tomcat

Tomcat server logs for each node.

Define with dse add-node --tomcat-logs=directory …​

Procedure

  1. Verify that your existing DSE installation has the default node configuration in the /etc/dse directory. The configuration files for the default node include /etc/dse/dse.yaml and /etc/dse/cassandra/cassandra.yaml.

  2. Give the default cluster a meaningful name. For example, change the default cluster named dse to payroll.

  3. Verify that the node binds to working IP addresses.

  4. Add DSE nodes to the DSE Multi-Instance cluster.

    • For package installations, you can use the dse add-node command. For example, to add a node that will join the cluster payroll on startup:

      sudo dse add-node <nodeId> --cluster payroll --listen-address <unused_ip_of_server>
--rpc-address <unused_ip_of_server> --seeds <ip_of_default_node>

    • For tarball installations, extract the <product>.tar.gz file multiple times and configure nodes in each location.

  5. Before starting the new node, set the node type in the /etc/default/<dse-nodeId> file:

    • DSE Search:

      SOLR_ENABLED=1

    • DSE Analytics:

      SPARK_ENABLED=1

  6. Continue configuring the node as appropriate.

    1. To change default DSE configuration values, edit the configuration files in /etc/nodeId.

      Ensure that the JMX port is configured for each node. 7199 is the DSE JMX metrics monitoring port. DataStax recommends allowing connections only from the local node. Configure SSL and JMX authentication when allowing connections from other nodes.

    2. To change default database configuration values, edit the /etc/<dse-nodeId>/cassandra/cassandra.yaml file.

    See DSE Multi-Instance file locations.

  7. After you make configuration changes, start the node.

    If the following error appears, see DSE cannot start with yaml parsing error.

    WARNING: Timed out while waiting for {product-short} to start.

  8. Verify that the nodes are running and are part of the cluster.

    For example, to verify the cluster status from a local node named dse-node1 on a DSE Multi-Instance cluster:

    sudo dse dse-node1 dsetool ring

    With DSE Multi-Instance, the output includes the Server ID:

    Server ID          Address      DC          Rack   Workload    Graph  Status  State    Load       Owns     VNodes                                       Health [0,1)
42-01-0A-F0-00-02  10.240.0.2   Cassandra   rack1  Cassandra   no     Up      Normal   92.13 KB   46.86%   -9223372036854775808                         0.17
42-01-0A-F0-00-02  127.0.0.1    Cassandra   rack1  Cassandra   no     Up      Normal   150.6 KB   53.14%   579561378715200106

    Using the standard dsetool ring command provides the status of the default node dse:

    sudo dsetool ring

    When a DSE Multi-Instance server is present in the cluster, the output always includes the Server ID column, even when you run the command on a server that is a DSE Multi-Instance host machine:

    Server ID          Address      DC          Rack   Workload    Graph  Status  State    Load       Owns     VNodes                                       Health [0,1)
42-01-0A-F0-00-02  10.240.0.2   Cassandra   rack1  Cassandra   no     Up      Normal   92.13 KB   46.86%   -9223372036854775808                         0.17
42-01-0A-F0-00-02  127.0.0.1    Cassandra   rack1  Cassandra   no     Up      Normal   150.6 KB   53.14%   579561378715200106

  9. To run standard DSE commands for nodes on a DSE Multi-Instance host machine, specify the node name using this syntax:

    sudo dse <dse-nodeId> <subcommand> [<command_arguments>]

    The node ID that is specified with the add-node command is automatically prefixed with dse-. In all instances except for add-node, the command syntax requires the dse- prefix.

    For example, with DSE Multi-Instance, the command to start a Spark shell on a node named dse-spark-node is:

    sudo dse dse-spark-node spark

    In contrast, the command to start a Spark shell without DSE Multi-Instance is:

    dse spark

See also

Was this helpful?

Give Feedback

How can we improve the documentation?

© Copyright IBM Corporation 2026 | Privacy policy | Terms of use |  Manage Privacy Choices

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: Contact IBM