Phases of the Zero Downtime Migration process
With Zero Downtime Migration, your applications can continue to run while you migrate data from one Cassandra-based cluster to another, resulting in little or no downtime and minimal service interruptions.
Why migrate?
There are many reasons that you might need to migrate data and applications. For example:
-
You want to move to a different database provider. For example, you might move from self-managed clusters to a cloud-based Database-as-a-Service (DBaaS), such as Astra DB.
-
You need to upgrade a cluster to a newer version or infrastructure.
-
You want to move client applications from shared clusters to dedicated clusters for greater control over individual configurations.
-
You want to consolidate client applications running on separate clusters onto one shared cluster to minimize sprawl and maintenance.
ZDM is comprised of ZDM Proxy, ZDM Utility, and ZDM Proxy Automation, which orchestrate activity-in-transition on your clusters. To move and validate data, you use Astra DB Sideloader, Cassandra Data Migrator, or DSBulk Migrator. ZDM Proxy keeps your clusters in sync at all times by a dual-write logic configuration, which means you can stop the migration or roll back at any point. For more information about these tools, see Compare DataStax migration tools.
When the migration is complete, the data is present in the new database, and you can update your client applications to connect exclusively to the new database. The old database becomes obsolete and can be removed.
ZDM requirements
True zero downtime migration is only possible if your database meets the minimum requirements described in Feasibility checks. If your database doesn’t meet these requirements, you can still complete the migration, but downtime might be necessary to finish the migration.
Supported migration paths
You can use ZDM Proxy to support migrations from Apache Cassandra®, DataStax Enterprise (DSE), Hyper-Converged Database (HCD), Astra DB, and other Cassandra-based databases to any other Cassandra-based database of the equivalent type or version:
- Compatible origin clusters
-
Migrate from one of the following:
-
DataStax Enterprise (DSE) version 4.7.1 and later
-
Apache Cassandra® version 2.1.6 and later
-
Other Cassandra-based databases that are based on a compatible Cassandra version, such as Astra DB Classic, ScyllaDB, and Yugabyte.
-
- Compatible target clusters
-
Migrate to one of the following:
-
A cluster running the same or later version of Cassandra or DSE
-
For more Astra DB migration paths, see Astra Migration Toolkit.
Migration phases
A migration project includes preparation for the migration and five migration phases.
The following sections describe the major events in each phase and how your client applications perform read and write operations on your origin and target clusters during each phase.
The origin is is your existing Cassandra-based environment, which can be Apache Cassandra, DSE, or Astra DB. The target is your new Cassandra-based environment where you want to migrate your data and client applications.
Pre-migration client application operations
Here’s a look at a pre-migration from a high-level view. At this point, your client applications are performing read/write operations with an existing CQL-compatible database such as Apache Cassandra, DSE, or Astra DB.
|
For the migration to succeed, the origin and target clusters must have matching schemas. A CQL statement that your client application sends to ZDM Proxy must be able to succeed on both the origin and target clusters. This means that any keyspace that your client application uses must exist on both the origin and target clusters with the same name. The table names, column names, and data types must also match. For more information, see Schema/keyspace compatibility. |
Migration planning
Before you begin the migration, plan and prepare for the migration:
Phase 1: Deploy ZDM Proxy and connect client applications
In this first phase, deploy the ZDM Proxy instances and connect client applications to the proxies. This phase activates the dual-write logic. Writes are bifurcated (sent to both the origin and target), while reads are executed on the origin only.
Phase 2: Migrate data
In this phase, migrate existing data using Astra DB Sideloader, Cassandra Data Migrator, or DSBulk Migrator. For information about these tools, see Compare DataStax migration tools.
ZDM Proxy will continue to perform dual writes while you move data and validate that the migrated data is correct.
Phase 3: Enable asynchronous dual reads
In this phase, you can optionally enable asynchronous dual reads. The idea is to test performance and verify that the target cluster can handle your application’s live request load before cutting over from the origin to the target permanently.
Phase 4: Route reads to the target cluster
In this phase, read routing on the ZDM Proxy is switched to teh target cluster so that all reads are executed on the target. Writes are still sent to both clusters.
At this point, the target becomes the primary cluster.
Phase 5: Connect directly to the target cluster
In this phase, move your client applications off the ZDM Proxy and connect them directly to the target cluster.
Once this happens, the migration is complete, and you now exclusively use the target cluster.
Zero Downtime Migration interactive lab
As a companion to the ZDM documentation, you can use the Zero Downtime Migration interactive lab to try the entire migration process in a demo environment.
The lab only requires a GitHub account and a supported browser. All browsers are supported except Safari.
You don’t need to install anything because the lab uses a pre-configured GitPod environment.
This lab provides an interactive, detailed walkthrough of the migration process, including pre-migration preparation and each of the five migration phases. The lab describes and demonstrates all steps and automation required to prepare for and complete a migration from any supported origin database to any supported target database.
