Phase 3: Enable asynchronous dual reads
After migrating and validating your data in Phase 2, you can begin to test your target cluster’s production readiness.
Phase 3 is optional but highly recommended. In this phase, you enable the asynchronous dual reads feature to test the secondary (target) cluster’s ability to handle a production workload before you permanently redirect read requests in Phase 4.
By default, ZDM Proxy sends all reads to the primary (origin) cluster, and then returns the result to the client application.
When you enable asynchronous dual reads, ZDM Proxy sends asynchronous read requests to the secondary cluster in addition to the synchronous read requests that are sent to the primary cluster.
At this point in the migration process, the secondary cluster is typically the target cluster. Because this feature is intended to test your target cluster’s ability to handle a simulated production workload, there is no reason to run it against the origin cluster that is already capable of handling production workloads.
This allows you to assess the target cluster’s performance and make any adjustments before permanently switching your workloads to the target cluster.
Response and error handling with asynchronous dual reads
With or without asynchronous dual reads, the client application only receives results from synchronous reads on the primary cluster. The client never receives results from asynchronous reads on the secondary cluster because these results are used only for ZDM Proxy’s asynchronous dual read metrics and testing purposes.
By design, if an asynchronous read fails or times out, it has no impact on client operations and the client application doesn’t receive an error. However, the increased workload from read requests can cause write requests to fail or time out on the secondary cluster. With or without asynchronous dual reads, a failed write on either cluster returns an error to the client application and potentially triggers a retry.
This functionality is intentional so you can simulate production-scale read traffic on the secondary cluster, in addition to the existing write traffic from ZDM Proxy’s dual writes, with the least impact to your applications.
To avoid unnecessary failures due to missing unmigrated data, don’t enable asynchronous dual reads until you migrate, validate, and reconcile all data from the origin cluster to the target cluster.
Configure asynchronous dual reads
Use the read_mode variable to enable or disable asynchronous dual reads.
Then, perform a rolling restart of your ZDM Proxy instances to apply the configuration change.
-
Edit
vars/zdm_proxy_core_config.yml, and then set theread_modevariable:-
Enable asynchronous dual reads
-
Disable asynchronous dual reads (default)
read_mode: DUAL_ASYNC_ON_SECONDARYread_mode: PRIMARY_ONLY -
-
Perform a rolling restart to apply the configuration change to your ZDM Proxy instances.
Monitor the target cluster’s performance
After enabling asynchronous dual reads, observe the target cluster’s performance to determine how well it performs under the expected production workload.
To assess performance, you can monitor the following:
-
Cluster health metrics like latency, throughput, and error rate
-
ZDM Proxy’s asynchronous read requests metrics
If needed, adjust the target cluster’s configuration and continue monitoring until the cluster reaches your performance targets.
If read requests fail due to missing data, go back to Phase 2 and repeat your data validation and reconciliation processes as needed to rectify the missing data errors.
If your data model includes non-idempotent operations, ensure that this data is handled correctly during data migration, reconciliation, and ongoing dual writes. For more information, see Lightweight Transactions and other non-idempotent operations.
Next steps
|
Don’t rush through this phase. Take time to verify that the target cluster can handle the expected production workloads successfully before proceeding to Phase 4. Make adjustments and reassess performance until you are confident in the cluster’s readiness. Asynchronous dual reads simulate real read requests without impacting your applications or end users. However, in Phase 4, genuine production read requests are directed to the target cluster exclusively. If your target cluster hasn’t been fully tested and prepared for production workloads, your applications can experience performance degradation, timeouts, and other issues. |
When you are confident that the target cluster is prepared to handle production workloads, you can disable asynchronous dual reads, and then proceed to Phase 4 where you will permanently route read requests to the target cluster.
