Compatibility requirements for ZDM Proxy

ZDM Proxy supports protocol V2, V3, V4, V5, DSE_V1, and DSE_V2. However, your clusters might not support these protocol versions.

When a specific protocol version is requested, ZDM Proxy handles protocol negotiation to ensure the requested version is supported by both clusters. For example, to use protocol V5 with ZDM Proxy, both the origin and target clusters must support V5. This includes HCD and open source Apache Cassandra® 4.0 or later. Otherwise, a lower protocol version must be used.

If the requested version isn’t mutually supported, then ZDM Proxy can force the client application to downgrade to a mutually supported protocol version. If automatic forced downgrade isn’t possible, then the connection fails, and you must modify your client application to request a different protocol version.

To successfully use the ZDM Proxy and migrate with no downtime, your origin (source) and target (destination) clusters must be compatible.

You can use ZDM Proxy to support migrations between Astra, DataStax Enterprise (DSE), Hyper-Converged Database (HCD), open-source Apache Cassandra, and and other Cassandra-based databases. Supported migration paths include cross-platform migrations and same-platform upgrades.

ZDM Proxy doesn’t modify or transform CQL statements with the exception of optional replacement for now().

A CQL statement that your client application sends to ZDM Proxy must be able to succeed on both clusters. This means that any keyspace that your client application uses must exist on both the origin and target clusters with the same keyspace name, table names, column names, and compatible data types. However, the keyspaces can have different replication strategies and durable write settings.

At the column level, the schema doesn’t need to be a one-to-one match as long as the CQL statements can be executed successfully on both clusters. For example, if a table has 10 columns, and your client application uses only five of those columns, then you can re-create the table on the target cluster with only the five required columns. However, the data from the other columns won’t be migrated to the target cluster if those columns don’t exist on the target cluster. Before you decide to omit a column from the target cluster, make sure that it is acceptable to permanently lose that data after the migration.

In some cases, you can change the primary key on the target cluster. For example, if your compound primary key is PRIMARY KEY (A, B), and you always provide parameters for the A and B columns in your CQL statements, then you could change the key to PRIMARY KEY (B, A) when creating the schema on the target because your CQL statements will still run successfully.

See How ZDM Proxy handles reads and writes.

Non-idempotent CQL operations don’t produce the exact same result each time the request is executed. Whether the request is executed and the actual value written to the database depend on the column’s current value or other conditions at the time of the request. Examples of non-idempotent operations include the following:

By design, a ZDM migration involves two separate clusters, and it is possible for the clusters to have differences due to non-idempotent operations.

If you use non-idempotent operations, DataStax recommends adding a reconciliation phase to your migration before and after Phase 4. This allows you an additional opportunity to resolve any data inconsistencies that are produced by non-idempotent operations. CDM is ideal for detecting and reconciling these types of inconsistencies.

However, if your application workloads can tolerate inconsistencies produced by LWTs and non-idempotent operations, you might not need to perform any additional validation or reconciliation steps. This depends entirely on your application business logic and requirements. It is your responsibility to determine whether your workloads can tolerate these inconsistencies and to what extent.

Throughout the migration process, you must restart ZDM Proxy instances to apply configuration changes. Client applications handle this in the same way as typical rolling restarts of Cassandra-based clusters.

If your applications already tolerate rolling restarts of your origin cluster, then you shouldn’t expect any issues during rolling restarts of ZDM Proxy instances.

To ensure that your client application retries requests that fail due to connection errors caused by the rolling restart process, check your driver’s retry policies and whether your requests are marked as idempotent. Most drivers treat all statements as non-idempotent by default, and they don’t automatically retry them. This means that you must explicitly mark statements as idempotent to trigger retries after connection errors. For more information, see Query idempotence in Cassandra drivers and Retry policies for Cassandra drivers.

The binary protocol used by Cassandra-based databases supports optional compression of transport-level requests and responses that reduces network traffic at the cost of CPU overhead.

When establishing connections from client applications, ZDM Proxy responds with a list of compression algorithms supported by both clusters. The compression algorithm configured in your Cassandra driver must match any item from the common list, or CQL request compression must be disabled completely. ZDM Proxy cannot decompress and recompress CQL requests using different compression algorithms.

This isn’t related to storage compression, which you can configure on specific tables with the compression table property. Storage/table compression doesn’t affect the client application or ZDM Proxy in any way.

Token-aware routing isn’t enforced when connecting through ZDM Proxy because these instances don’t hold actual token ranges in the same way as database nodes. Instead, each ZDM Proxy instance has a unique, non-overlapping set of synthetic tokens that simulate token ownership and enable balanced load distribution across the instances.

Upon receiving a request, a ZDM Proxy instance routes the request to appropriate origin and target database nodes, independent of token ownership.

If your clients have token-aware routing enabled, you don’t need to disable this behavior while using ZDM Proxy. Clients can continue to operate with token-aware routing enabled without negative impacts to functionality or performance.

A cluster’s authorizer doesn’t affect client applications or ZDM Proxy, which means that you can use any kind of authorizer configuration on your clusters, and they can use different authorizers.

In contrast, a cluster’s authenticator must be compatible with ZDM Proxy. ZDM Proxy supports the following cluster authenticator configurations:

ZDM Proxy doesn’t support DseAuthenticator with kerberos scheme.

The origin and target clusters can have different authentication configurations because ZDM Proxy treats them independently.

If your migration includes a DSE cluster that supports DSE Search or DSE Graph workloads, it is important that you understand how ZDM Proxy handles DSE advanced workloads:

For more DSE-specific migration considerations, see Migrate to DataStax Enterprise (DSE).

Complex migration scenarios, such as multi-datacenter migrations and many-to-one migrations, require additional planning to properly configure migration tools and effectively migrate the data.

Factors that can make a migration more complicated include the following:

Multi-region migrations, also known as multi-datacenter migrations, can include one or more of the following scenarios:

For relatively small migrations to DSE with consistent data across all regions, you can migrate the primary region first, and then deploy additional datacenters after the initial migration is complete, using an appropriate replication strategy to copy the data to the additional datacenters. However, this approach isn’t suitable for all migrations because replication strategies, like eventual consistency, can be inefficient when replicating large amounts of data across geographically distant regions.

It is difficult to provide a one-size-fits-all solution for multi-region migrations due to the potential complexity and variability of these scenarios. For assistance with your migration, contact your DataStax account representative or IBM Support.

To configure ZDM Proxy for a multi-datacenter migration, see the ZDM Proxy infrastructure guidelines for multi-datacenter clusters.

Next, prepare the ZDM Proxy infrastructure.