DataStax Zero Downtime Migration Release Notes

ZDM Proxy Automation 2.3.0 update

03 February 2023

Released ZDM Proxy Automation 2.3.0, which enables ansible scripts and terraform to work with both Ubuntu and RedHat-family Linux distributions. Documentation updates included the following in the Machines section of the Deployment and infrastructure considerations topic:

"Ubuntu Linux 20.04 or newer, RedHat Family Linux 7 or newer"

ZDM Proxy Automation 2.2.0 update

31 January 2023

Starting in version 2.2.0 of the ZDM Proxy Automation, we added the zdm_proxy_cluster_config.yml file to contain all the configuration variables for Origin and Target. Prior to version 2.2.0, the variables were in the zdm_proxy_core_config.yml file.

This change is backward compatible. If you previously populated the variables in zdm_proxy_core_config.yml, these variables will be honored and take precedence over any variables in zdm_proxy_cluster_config.yml, if both files are present.

We encourage existing 2.x ZDM users to upgrade to the 2.3.0 version of ZDM Proxy Automation. To do so, simply git pull the main branch of https://github.com/datastax/zdm-proxy-automation from within the Ansible Control Host container. You can also check out a specific tag, such as 2.3.0.

For more about the YML files used to configure access to your clusters, see this topic.

The latest ZDM Proxy version is still 2.1.0. The latest ZDM Proxy Automation version is 2.3.0.

If you are using a ZDM Proxy Automation version up to and including 2.1.0, please use zdm_proxy_core_config.yml to configure access to your clusters.

ZDM 2.1.0 release

13 January 2023

The ZDM 2.1.0 release adds ZDM Proxy heartbeat functionality and provides several bug fixes.

The periodic heartbeat feature in 2.1.0 has been implemented to keep alive idle cluster connections.

By default, ZDM Proxy now sends heartbeats after 30 seconds of inactivity on a cluster connection. You can tune the heartbeat interval with the Ansible configuration variable heartbeat_insterval_ms, or by directly setting the ZDM_HEARTBEAT_INTERVAL_MS environment variable if you do not use the ZDM Proxy Automation.

DataStax strongly recommends that you use version 2.1.0 (or newer) to benefit from this improvement, especially if you have a read-only workload.

To verify which ZDM Proxy version you’re running, see this topic.

To find out how to upgrade an existing ZDM Proxy deployment, see Upgrade the proxy version.

ZDM Proxy 2.1.0 changes

For the latest information about ZDM Proxy new features and other changes, please refer to these GitHub-hosted documents in the open-source ZDM Proxy repo:

ZDM 2.1.0 documentation updates

The following topics have been updated for the 2.1.0 release:

Feasibility checks for read-only applications. See the notes indicating that this issue is solved by the ZDM Proxy 2.1.0 release.
Change a mutable configuration variable. See the heartbeat_interval_ms and zdm-proxy_max_stream_ids information.
Async read timeouts. See the clarification in the Workaround section indicating that this issue is solved by the ZDM Proxy 2.1.0 release.
Node-level metrics. See the "Number of Used Stream Ids" section.

ZDM 2.0.0 release

18 October 2022

ZDM Proxy 2.0.0 changes

This 2.0.0 version marks the public release of the self-service DataStax Zero Downtime Migration product suite.

The following GitHub repos are public. You are welcome to read the source and submit feedback via GitHub Issues per repo.

ZDM Proxy open-source repo: in addition to sending feedback, you may submit Pull Requests (PRs) for potential inclusion, provided you accept the DataStax Contributor License Agreement (CLA). For more information, see Contribution guidelines.
ZDM Proxy Automation repo for Ansible-based ZDM Proxy automation.
DSBulk Migrator repo for migration of smaller data quantities.
Cassandra Data Migrator repo for migration of larger data quantities and where detailed verifications and reconciliation options are needed.

This suite of tools allows for zero downtime migration only if your database meets the minimum requirements. If your database does not meet these requirements, you can complete the migration from Origin to Target, but downtime might be necessary to finish the migration.

For the latest information about ZDM Proxy new features and other changes, please refer to the GitHub-hosted RELEASE_NOTES in the open-source ZDM Proxy repo. The document includes CHANGELOG links for each ZDM Proxy N.n release.

The Zero Downtime Migration process requires you to be able to perform rolling restarts of your client applications during the migration. This is standard practice for client applications that are deployed over multiple instances and is a widely used approach to roll out releases and configuration changes.

ZDM 2.0.0 documentation updates

Starting with the 2.0.0 version on 18-Oct-2022, the Zero Downtime Migration documentation set is available online, starting here.

Supported releases

Overall, you can use ZDM Proxy to migrate:

From: Any Cassandra 2.1.6 or higher release, or from any DSE 4.7.1 or higher release
To: Any equivalent or higher release of Cassandra, or to any equivalent or higher release of DSE, or to Astra DB

Migration scenarios

There are many reasons why you may decide to migrate your data and client applications from one cluster to another, for example:

Moving to a different type of CQL database, for example an on-demand cloud-based proposition such as Astra DB.
Upgrading a cluster to a newer version, or newer infrastructure, in as little as one step while leaving your existing cluster untouched throughout the process.
Moving one or more client applications out of a shared cluster and onto a dedicated one, in order to manage and configure each cluster independently.
Consolidating client applications, which may be currently running on separate clusters, onto a shared one in order to reduce overall database footprint and maintenance overhead.

Here are just a few examples of migration scenarios that are supported when moving from one type of CQL-based database to another:

From an existing self-managed Cassandra or DSE cluster to cloud-native Astra DB. For example:
- Cassandra 2.1.6+, 3.11.x, 4.0.x, or 4.1.x to Astra DB
- DSE 4.7.1+, 4.8.x, 5.1.x, 6.7.x or 6.8.x to Astra DB
From an existing Cassandra or DSE cluster to another Cassandra or DSE cluster. For example:
- Cassandra 2.1.6+ or 3.11.x to Cassandra 4.0.x or 4.1.x
- DSE 4.7.1+, 4.8.x, 5.1.x or 6.7.x to DSE 6.8.x
- Cassandra 2.1.6+, 3.11.x, 4.0.x, or 4.1.x to DSE 6.8.x
- DSE 4.7.1+ or 4.8.x to Cassandra 4.0.x or 4.1.x
From Astra DB Classic to Astra DB Serverless
From any CQL-based database type/version to the equivalent CQL-based database type/version.