Zero Downtime Migration frequently asked questions
This page includes common questions about the DataStax Zero Downtime Migration (ZDM) tools.
What is a zero-downtime migration?
A zero-downtime migration with the DataStax ZDM tools means you can reliably migrate your client applications and data between CQL clusters with no interruption of service.
Why should I use the ZDM tools for my migration?
There are several benefits to using the ZDM tools for your migration:
- Minimal client code changes
-
Depending on cluster compatibility, the ZDM tools help you migrate to a new or upgraded database platform with minimal changes to your client application code. In some cases, you only need to change the connection string to point to the new cluster at the end of the migration process. Typically, these changes are minimal and non-invasive, especially if your client application uses an externalized property configuration for contact points.
- Real-time data consistency
-
ZDM Proxy orchestrates real-time activity generated by your client applications, ensuring data consistency while you replicate, validate, and test your existing data on the new cluster. Once you set up ZDM Proxy, the dual-writes feature ensures that new writes are sent to both the origin and target clusters, so you can focus on migrating the data that was present before initializing ZDM Proxy.
- Safely test the new cluster under full production workloads
-
In addition to the dual-writes feature, you can optionally enable asynchronous dual-reads to test the target cluster’s ability to handle a production workload before you permanently switch to the target cluster at the end of the migration process.
Client applications aren’t interrupted by read errors or latency spikes on the new, target cluster. Although these errors and metrics are received by ZDM Proxy for monitoring and performance benchmarking purposes, they aren’t propagated back to the client applications.
From the client side, traffic is seamless and uninterrupted during the entire migration process.
- Seamless rollback without data loss
-
If there is a problem during the migration, you can rollback to the original cluster without any data loss or interruption of service. You can allow ZDM Proxy to continue orchestrating dual-writes, or redirect your client applications back to the origin cluster and disable ZDM Proxy.
- Endless validation and testing time
-
Because your client applications remain fully operational during the migration, and your clusters are kept in sync by ZDM Proxy, you can take as much time as you need to validate and test the target cluster before switching over permanently.
- Migrate to a different platform or perform major version upgrades
-
The ZDM tools support migrations between different CQL-based platforms, such as open-source Apache Cassandra® to Astra DB, as well as major version upgrades of the same platform, such as DSE 5.0 to DSE 6.9.
Is there a ZDM demo?
Yes, you can use the ZDM interactive lab to see how the migration process works.
What ZDM tools are available?
The ZDM tools are ZDM Proxy, ZDM Utility, and ZDM Proxy Automation. These tools orchestrate the traffic between your client applications and the origin and target clusters during the migration process.
For the actual data migration, there are many tools you can use, such as Astra DB Sideloader, Cassandra Data Migrator, DSBulk Migrator, and custom data migration scripts.
For more information, see Compare DataStax migration tools.
What is ZDM Proxy?
Generally speaking, a proxy is a software class functioning as an interface to any other component, connection, or resource, such as a network connection, a server, a large object in memory, or a file. The proxy is a wrapper or agent object that the client calls to access the real object served through that proxy.
In the context of ZDM, the ZDM Proxy is an open-source component designed to seamlessly handle real-time client application activity while a migration is in progress. For more information, see Compare DataStax migration tools.
What are ZDM Proxy Automation and ZDM Utility?
ZDM Proxy Automation is an Ansible-based tool that allows you to deploy and manage multiple ZDM Proxy instances and the associated monitoring stack. To simplify the setup, the ZDM Proxy Automation suite includes ZDM Utility, which is an interactive utility that creates a Docker container to act as the Ansible Control Host. For more information, see Compare DataStax migration tools.
Does the ZDM process migrate clusters?
The ZDM process doesn’t directly migrate clusters. Instead, the ZDM tools orchestrate live traffic between your existing cluster and a new cluster while you use a data migration tool to replicate and validate data on the new cluster.
ZDM Proxy handles real-time requests generated by your client applications during the migration process, and keeps both clusters in sync through dual writes. If there is a problem during the migration, you can confidently rollback to the original cluster without data loss or interruption of service.
At the end of the migration process, your client application connects exclusively to your new cluster, and then you decommission ZDM Proxy and the old cluster.
What challenges does ZDM solve?
Before the ZDM tools were available, migrating client applications between clusters involved granular and intrusive client application code changes, extensive migration preparation, and a window of downtime for the client application’s end users.
With the ZDM tools, you can migrate your client applications and data between CQL clusters with minimal code changes and no interruption of service. You can have the confidence that you are using tools designed specifically to handle the complexities of live traffic during large enterprise migrations.
What is the pricing model?
ZDM Proxy, ZDM Utility, ZDM Proxy Automation, Cassandra Data Migrator, and DSBulk Migrator are free and open-sourced.
Astra DB Sideloader is part of an Astra DB Enterprise subscription plan, and it incurs costs based on usage.
Where can I get help with my migration?
Technical assistance with the ZDM process is available from DataStax Support for DSE users, IBM Elite Support for Apache Cassandra subscribers, and Astra organizations on an Enterprise plan.
For any observed problems with ZDM Proxy or the other open-source ZDM and data migration tools, you can report an issue in their respective GitHub repositories:
-
ZDM Proxy Automation repository (includes ZDM Proxy Automation and ZDM Utility)
Does ZDM Proxy support Transport Layer Security (TLS)?
Yes, see Configure Transport Layer Security (TLS).
Can ZDM Proxy be deployed as a sidecar?
Don’t deploy ZDM Proxy as a sidecar. For more information, see Choosing where to deploy the proxy.
What are origin, target, primary, and secondary clusters?
These terms refer to where your data is located during the migration process, and where read and write requests are sent by ZDM Proxy:
- Origin and target
-
The origin cluster is your existing database that you are migrating away from. The target cluster is your new database that you are migrating to.
Origin and target cluster credentials are provided to ZDM Proxy so it can establish connections and send requests to both clusters.
- Primary and secondary
-
The primary cluster is the database that is designated as the source of truth for read requests. It receives all read requests by default, and the responses from these read requests are returned to the client application. The primary cluster is set by ZDM Proxy Automation through the
primary_clustervariable, or you can set it directly through the ZDM ProxyZDM_PRIMARY_CLUSTERenvironment variable.The other database is the secondary cluster. It doesn’t receive read requests unless you enable asynchronous dual-reads.
For the majority of the migration process, the origin cluster is the primary cluster, and the target cluster is the secondary cluster. Near the end of the migration, when you have validated that all pre-existing data has been replicated to the target cluster, you set the target cluster as the primary cluster.
Throughout the entire migration, until you decommission ZDM Proxy, both clusters receive all write requests through the dual writes feature. For more information, see How ZDM Proxy handles reads and writes.
Is there a difference between cluster and database?
In the context of the ZDM process, the terms cluster and database are used interchangeably to refer to the source and destination for the data that you are moving during your migration.
