Phases of the Zero Downtime Migration process
With Zero Downtime Migration, your applications can continue to run while you migrate data from one Cassandra-based database to another, resulting in little or no downtime and minimal service interruptions.
Why migrate?
There are many reasons that you might need to migrate data and applications. For example:
-
You want to move to a different database provider. For example, you might move from self-managed clusters to a cloud-based Database-as-a-Service (DBaaS), such as Astra DB.
-
You need to upgrade a cluster to a newer version or infrastructure.
-
You want to move client applications from shared clusters to dedicated clusters for greater control over individual configurations.
-
You want to consolidate client applications running on separate clusters onto one shared cluster to minimize sprawl and maintenance.
ZDM is comprised of ZDM Proxy, ZDM Utility, and ZDM Proxy Automation, which orchestrate activity-in-transition on your databases. To move and validate data, you use Astra DB Sideloader, Cassandra Data Migrator, or DSBulk Migrator. ZDM Proxy keeps your databases in sync at all times by a dual-write logic configuration, which means you can stop the migration or roll back at any point. For more information about these tools, see Compare DataStax migration tools.
When the migration is complete, the data is present in the new database, and you can update your client applications to connect exclusively to the new database. The old database becomes obsolete and can be removed.
Requirements for zero downtime
True zero downtime migration is only possible if your database meets the minimum requirements, including cluster compatibility, described in Feasibility checks If your database doesn’t meet these requirements, you can still complete the migration, but downtime might be necessary to finish the migration.
For more information, see Feasibility checks
Migration phases
A migration project includes preparation for the migration and five migration phases.
The following sections describe the major events in each phase and how your client applications perform read and write operations on your origin and target databases during each phase.
The origin is is your existing Cassandra-based environment, which can be Apache Cassandra, DSE, or Astra DB. The target is your new Cassandra-based environment where you want to migrate your data and client applications.
Migration planning
Before you begin a migration, your client applications perform read/write operations with your existing CQL-compatible database, such as Apache Cassandra, DSE, HCD, or Astra DB.
While your application is stable with the current data model and database platform, you might need to make some adjustments before enabling ZDM Proxy.
|
For the migration to succeed, the origin and target databases must have matching schemas, including keyspace names, table names, column names, and data types. A CQL statement that your client application sends to ZDM Proxy must be able to succeed on both databases. For more information, see Schema/keyspace compatibility. |
Before you begin the migration, plan and prepare for the migration by setting up your target infrastructure, reviewing compatibility requirements for ZDM Proxy, and understanding when you can rollback the migration if necessary:
Phase 1: Deploy ZDM Proxy and connect client applications
In this first phase, deploy the ZDM Proxy instances and connect client applications to the proxies. This phase activates the dual-write logic. Writes are sent to both the origin and target databases, while reads are executed on the origin only.
For more information and instructions, see Phase 1: Deploy ZDM Proxy and connect client applications.
Phase 2: Migrate data
In this phase, you use a data migration tool to copy your existing data to the target database. ZDM Proxy continues to perform dual writes so that you can focus on moving data that was present before you connected ZDM Proxy. Then, you thoroughly validate the migrated data, resolving missing and mismatched records, before moving on to the next phase.
For more information and instructions, see Migrate and validate data.
Phase 3: Enable asynchronous dual reads
This phase is optional but recommended.
In this phase, you can enable the asynchronous dual reads feature to test the target database’s ability to handle a production workload before you permanently switch your applications to the target database at the end of the migration process.
When enabled, ZDM Proxy sends asynchronous read requests to the secondary database in addition to the synchronous read requests that are sent to the primary database by default.
For more information, see Enable asynchronous dual reads and How ZDM Proxy handles reads and writes.
Phase 4: Route reads to the target database
In this phase, read routing on ZDM Proxy is switched to the target database so that all reads are executed on the target. Writes are still sent to both databases in case you need to rollback the migration.
At this point, the target database becomes the primary database.
For more information and instructions, see Route reads to the target.
Phase 5: Connect directly to the target database
In the final phase of the migration, you move your client applications off ZDM Proxy and connect them directly to the target database.
Once this happens, the migration is complete, and you now exclusively use the target database.
Whether you choose to destroy to retain the origin database depends on your organization’s policies and whether you might need to revert to it in the future. However, be aware that the origin database is no longer synchronized with the target database, and the origin database won’t contain writes that happen after you disconnect ZDM Proxy.
For more information, see Phase 5: Connect your client applications directly to the target.
Zero Downtime Migration interactive lab
As a companion to the ZDM documentation, you can use the Zero Downtime Migration interactive lab to try the entire migration process in a demo environment.
The lab only requires a GitHub account and a supported browser. All browsers are supported except Safari.
You don’t need to install anything because the lab uses a pre-configured GitPod environment.
This lab provides an interactive, detailed walkthrough of the migration process, including pre-migration preparation and each of the five migration phases. The lab describes and demonstrates all steps and automation required to prepare for and complete a migration from any supported origin database to any supported target database.
