Migrate a DSE Cluster to DataStax Mission Control
DataStax Mission Control is currently in Public Preview. DataStax Mission Control is not intended for production use, has not been certified for production workloads, and might contain bugs and other functional issues. There is no guarantee that DataStax Mission Control will ever become generally available. DataStax Mission Control is provided on an “AS IS” basis, without warranty or indemnity of any kind. If you are interested in trying out DataStax Mission Control please join the Public Preview. |
Migrating an existing DSE cluster to DataStax Mission Control with zero down time is trivial and allows for the management of existing clusters without moving the data to new hardware.
Prerequisities and system requirements
-
DataStax Mission Control Runtime must be installed prior to migration. See Services setup with DataStax Mission Control Runtime Installer.
-
Each node must have the same storage group rights in the DSE data directories (ownership).
-
Group read-write access must be given to DSE directories or the migration cannot continue. DataStax Mission Control changes file system groups and write access during migration.
-
-
User running DataStax Mission Control
mcctl
commands must have access to the DSE’s data directories (running withsudo -u
is an acceptable solution). -
Only JDK8 is supported as the source and target platform.
-
Linux is the only supported platform.
-
Kubernetes must be installed on every DSE node before running the migration.
-
Only DSE server versions 6.8.26 and later are supported. Check your version with:
nodetool version
Procedure
The migration process consists of three phases; import init
, import add
, and import commit
.
-
The
import init
phase validates the cluster and gathers the necessary configuration data to be used as the base to determine if the migration is possible. It also creates the necessary seed services, the project in which the cluster is created, and related metadata so that migrated nodes can continue communicating with the non-migrated ones, and ensuring that the cluster stays healthy during the process. Run theimport init
phase only once on the first DSE node. -
The
import add
phase runs on every DSE node in the datacenter. Theimport add
command performs the per-node migration, using the configuration fetched in theimport init
phase.import add
supports running in parallel and takes care of serializing the migrations. -
The final
import commit
phase requires that theimport add
phase be run on every node in the cluster. It creates the necessary Kubernetes configurations. It activates all the migrated nodes to DataStax Mission Control and then reconciles them to the desired state. After achieving this state, the cluster and all of its nodes should be managed using DataStax Mission Control.
As you step through these phases, here are some useful Command Line Interface (CLI) flags for reference:
- --nodetool-path
-
Where the
nodetool
command is located. This assumes a complete path including thenodetool
executable.cqlsh
must reside in the same path. - --nodetool status
-
Describes status and information about nodes in a datacenter.
- --cass-config-dir
-
Directory where
cassandra.yaml
is located. - --dse-config-dir
-
Directory where
dse.yaml
is located. - --cassandra-home
-
Directory where DSE is installed.
- --username
-
DSE admin account name.
- --password
-
DSE admin password.
- --jmxusername
-
Local JMX authentication username.
- --jmxpassword
-
Local JMX authentication password.
- --kubeconfig
-
Path to the kubeconfig file. If not specified, the default kubeconfig file is used.
- --namespace/-n
-
The project/namespace in which the cluster is imported/located. If not specified, the default namespace is used.
import init
phase
The import init
phase verifies that the cluster is ready to be migrated. It must be run from a node in the cluster because it connects to the running DSE node to fetch its state. JMX access credentials are required because this phase uses nodetool
. After verifying the node state, the configuration from this node is used as the base to which all of the other nodes in the managed cluster must conform. Choose a node with a good configuration to use as a generic base.
It is possible to review the configuration (or a failed run), modify the setup, and rerun the import init
command. Rerunning import init
replaces the values previously written to the cluster.
If the cluster is using authentication and you wish to use existing superuser
username/password combination for communication, then enter them using --username
and --password
parameters. When no credentials are given, DataStax Mission Control creates and instates its own superuser
to the cluster. Likewise, if the JMX local authentication is enabled, this DataStax Mission Control migration tool requires the JMX useraccount and password.
-
Run the
import init
command:mcctl import init -n dse
import add
phase
import add
is the process of migrating an existing DSE node to DataStax Mission Control. The DSE node must be running before continuing. You can run multiple nodes at the same with the import add
command. The command sequentially runs all the migrations, migrating a single node at a time.
The import add
command requires access to modify the data directories that the running DSE instance is using. Ownership and groups of directories must be unified prior to migration. When running under DataStax Mission Control, the nodes may use a user id or group id that differs from the existing deployment, thus requiring modification as part of migration.
The import add
command supports (and requires) the same parameters as does the import init
command if the installation is using a custom path instead of auto-detected paths.
When |
To aid in detection, the import add
command uses the DSE_HOME
environment variable. If either the cassandra-home
or the DSE_HOME
environment variable is set, DataStax Mission Control tries to detect cassandra.yaml
, dse.yaml
, and nodetool-path
from its subdirectories.
-
Run the
import add
command:sudo mcctl import add -n dse
The
import add
command sequentially deletes the DSE nodes and recreates them in a K8s cluster while the cluster remains up and running. -
Verify by running:
nodetool status
and
kubectl get pods -n dse
Reversing operation
At this point, the operation is reversible. This may be required if certain prerequisites are not met on every node and a particular node requires certain operations. The migration has not broken the existing DSE cluster installation, but you must remove the migrated node before continuing.
-
Delete the pod which this node created by running:
kubectl delete pod/<name> -n dse
-
List and then delete all the PersistentVolumeClaim (PVC)s and PersistentVolume (PV)s that this node mounted:
kubectl get pvc -n dse kubectl delete pvc <pvc-name> -n dse kubectl get pv |grep "pvc-server-data" kubectl delete pv <pv-name>
-
Modify the ClusterConfig
kubectl edit clusterconfig cluster-migrate-config -n dse
and set the
migrated
field tofalse
instead oftrue
. -
Restart the node using the OpsCenter procedure.
import commit
phase
Before running the import commit
phase, all the nodes must first be migrated. DataStax Mission Control checks every node’s migration status, and if any node is not migrated it exits in the validation phase. Upon successful validation, the final cluster configuration is changed and applied to all the migrated nodes. This causes a rolling restart of the cluster.
-
Run the
import commit
command:mcctl import commit -n dse
The
import commit
command creates the MissionControlCluster (mccluster
in short) object and finalizes the migration. -
Verify by running:
-
kubectl
command -
Sample result
kubectl get mccluster -n dse
NAMESPACE NAME AGE dse dsetest 73m
-
At the completion of these three phases the DSE cluster is fully migrated to DataStax Mission Control with zero down time.