Terminate a datacenter with Mission Control

You can use Mission Control to remove (terminate) datacenters in your deployed clusters.

To avoid cluster-wide downtime and instability, terminate datacenters in the following order based on workload types and deployment architecture:

For multi-cluster deployments, terminate datacenters from one cluster at a time.
For multi-datacenter clusters, terminate one datacenter at a time.

This restriction applies only when you plan to terminate more than one datacenter.
If using DSE advanced workloads (Analytics, Graph, Search), terminate datacenters according to the following workload type hierarchy:
1. Terminate DSE Analytics datacenters.
2. Terminate transactional (Cassandra) and DSE Graph datacenters.
3. Terminate DSE Search datacenters.
For mixed workload datacenters, assign priority by the highest ranking workload type in that datacenter. For example, a SearchAnalytics datacenter would be terminated before a Graph datacenter.

Termination workflow example

To demonstrate datacenter termination, this guide uses a multi-datacenter Kubernetes cluster with two datacenters and three nodes each (six total). The DC1 datacenter is in the west region, and the DC2 datacenter is in the east region. These examples terminate DC2 so that only DC1 remains.

The user updates the replication strategy on user-defined keyspaces to remove references on the east region datacenter being terminated.

You must wait until active keyspaces become dormant before altering the replication factor.

You must remove all references to the discontinued datacenter from all user-defined keyspaces.

By default, Mission Control operators automatically update the replication factors for all system keyspaces, and then terminate its nodes (pods). Mission Control issues an error, and termination fails if a user-defined keyspace is actively replicating to the datacenter that is targeted for termination.

This is a safety measure to prevent unintended removal of data.

The user submits the updated MissionControlCluster to the control plane Kubernetes cluster.
The cluster-level operator detects the modification, and then automatically updates keyspace replication settings on system keyspaces.
The cluster-level operator deletes datacenter-level resources in the Kubernetes cluster where the nodes will be terminated.
The datacenter-level operator detects the datacenter-level resource changes, and then deletes native Kubernetes objects representing the database nodes.

Terminate a datacenter within a deployed cluster

Modify the existing MissionControlCluster YAML (demo-dse.yaml) in the control plane Kubernetes cluster, updating the spec.datacenters list so that it no longer references the datacenter dc1 targeted for termination. In this example, the following lines are deleted:

spec:
  k8ssandra:
    cassandra:
      datacenters:
      - metadata:
          name: dc1
        k8sContext: east
        size: 3
        racks:
        - name: rack1
          nodeAffinityLabels:
          topology.kubernetes.io/zone: us-east1-c
        - name: rack2
          nodeAffinityLabels:
            topology.kubernetes.io/zone: us-east1-b
        - name: rack3
          nodeAffinityLabels:
            topology.kubernetes.io/zone: us-east1-d

Submit this modification to the Kubernetes control plane cluster with the following command:
```
kubectl apply -f demo-dse.cassandratask.yaml
```
The following keyspaces are updated:
- system_traces
- system_distributed
- system_auth
- dse_leases
- dse_perf
- dse_security
Monitor the termination operation progress in the control plane cluster with this command:
```
kubectl get k8ssandracluster demo
```
In the event that any user-defined keyspaces are still replicating to datacenter targeted for termination, the kubectl command returns an error such as:
```
NAME   ERROR
demo   cannot decommission DC dc1: keyspace ks1 still has replicas on it
```
To rectify this error, the replication of all user-defined keyspaces must be manually updated to remove references to the datacenter being terminated.
```
ALTER KEYSPACE ks1 WITH replication = {'class': 'NetworkTopologyStrategy', 'west': 3};
```
Monitor the termination progress by checking the status of the datacenter to be terminated in the east cluster. This example uses the following command:
```
kubectl get cassandradatacenter dc1 -o yaml
```
The sample output indicates that one node is online and one is not at this point in the monitoring. The CassandraDatacenter dc1 is terminated when DC-level operators set all of the Decommission conditions:status to "False". The nodeStatuses map is also updated.
```
status:
  cassandraOperatorProgress: Updating
  conditions:
  - lastTransitionTime: "2025-08-28T02:43:20Z"
    message: ""
    reason: ""
    status: "True"
    type: Healthy
  - lastTransitionTime: "2025-08-28T02:43:20Z"
    message: ""
    reason: ""
    status: "False"
    type: Stopped
...
```
The DC-level operators must terminate each node in the datacenter before the datacenter itself is terminated.

Monitor the logs with this command:

kubectl logs demo-dc1-rack3-sts-0 -c server-system-logger

where demo-dc1-rack3-sts-0 is the StatefulSet designation of the ordinal index of the node in a rack.

INFO  [pool-17-thread-1] 2025-08-28 17:13:09,717  StorageService.java:2143 - LEAVING: sleeping 30000 ms for batch processing and pending range setup
...
INFO  [pool-17-thread-1] 2025-08-28 17:13:39,770  Gossiper.java:1301 - InetAddress /10.100.5.15 is now DOWN
INFO  [pool-17-thread-1] 2025-08-28 17:13:39,788  StorageService.java:4968 - Announcing that I have left the ring for 30000ms

Upon successful completion of the east datacenter termination operation, users now run only in the west region datacenter.

Terminate a datacenter with Mission Control

Prerequisites

Termination order for cluster stability

Termination workflow example

Terminate a datacenter within a deployed cluster

Was this helpful?

Give Feedback