Adding a datacenter to a single-token architecture cluster

Steps for adding a datacenter to single-token architecture clusters, not clusters using Virtual nodes.

Only add new nodes to the cluster. A new node is a system in which DataStax Enterprise (DSE) has never started. The node must have absolutely NO PREVIOUS DATA in the data directory, saved_caches, commitlog, and hints. Adding nodes previously used for testing or that have been removed from another cluster, merges the older data into the cluster and may cause data loss or corruption.
Where is the cassandra-topology.properties file?

The location of the cassandra-topology.properties file depends on the type of installation:

Installation Type Location

Package installations + Installer-Services installations

/etc/dse/cassandra/cassandra-topology.properties

Tarball installations + Installer-No Services installations

<installation_location>/resources/cassandra/conf/cassandra-topology.properties
Where is the cassandra-rackdc.properties file?

The location of the cassandra-rackdc.properties depends on the type of installation:

Installation Type Location

Package installations + Installer-Services installations

/etc/dse/cassandra/cassandra-rackdc.properties

Tarball installations + Installer-No Services

<installation_location>/resources/cassandra/conf/cassandra-rackdc.properties
Where is the cassandra.yaml file?

The location of the cassandra.yaml file depends on the type of installation:

Installation Type Location

Package installations + Installer-Services installations

/etc/dse/cassandra/cassandra.yaml

Tarball installations + Installer-No Services installations

<installation_location>/resources/cassandra/conf/cassandra.yaml

Procedure

  1. Ensure that you are using NetworkTopologyStrategy for all keyspaces.

  2. For each new node, edit the configuration properties in the cassandra.yaml file:

    • Set auto_bootstrap to False.

    • Set the initial_token. Be sure to offset the tokens in the new datacenter, see Initializing single-token architecture datacenters.

    • Set the cluster name.

    • Set any other non-default settings.

    • Set the seed lists. Every node in the cluster must have the same list of seeds and include at least one node from each datacenter. Typically one to three seeds are used per datacenter.

  3. Update either the properties file on all nodes to include the new nodes. You do not need to restart.

    • GossipingPropertyFileSnitch: cassandra-rackdc.properties

    • PropertyFileSnitch: cassandra-topology.properties

  4. Ensure that your client does not auto-detect the new nodes so that they aren’t contacted by the client until explicitly directed.

  5. If using a QUORUM consistency level for reads or writes, check the LOCAL_QUORUM or EACH_QUORUM consistency level to make sure that the level meets the requirements for multiple datacenters.

  6. Start the new nodes.

  7. The GossipingPropertyFileSnitch always loads cassandra-topology.properties when that file is present. Remove the file from each node on any new cluster or any cluster migrated from the PropertyFileSnitch.

  8. After all nodes are running in the cluster:

    1. Change the replication factor for your keyspace for the expanded cluster.

    2. Run nodetool rebuild on each node in the new datacenter.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com