Phase 4: Change read routing to Target
This topic explains how you can configure the ZDM Proxy to route all reads to Target instead of Origin.
For illustrations of all the migration phases, see the Introduction.
Steps
You would typically perform these steps once you have migrated all the existing data from Origin, and completed all validation checks and reconciliation if necessary.
This operation is a configuration change that can be carried out as explained here.
|
If you performed the optional steps described in the prior topic, Phase 3: Enable asynchronous dual reads — to verify that your Target cluster was ready and tuned appropriately to handle the production read load — be sure to disable async dual reads when you’re done testing.
If you haven’t already, revert Otherwise, if you don’t disable async dual reads, ZDM Proxy instances would continue to send async reads to Origin, which, although harmless, is unnecessary. |
Changing the read routing configuration
If you’re not there already, ssh back into the jumphost:
ssh -F ~/.ssh/zdm_ssh_config jumphostOn the jumphost, connect to the Ansible Control Host container:
docker exec -it zdm-ansible-container bashYou will see a prompt like:
ubuntu@52772568517c:~$Now open the configuration file vars/zdm_proxy_core_config.yml for editing.
Change the variable primary_cluster to TARGET.
Run the playbook that changes the configuration of the existing ZDM Proxy deployment:
ansible-playbook rolling_update_zdm_proxy.yml -i zdm_ansible_inventoryWait for the ZDM Proxy instances to be restarted by Ansible, one by one. All instances will now send all reads to Target instead of Origin. In other words, Target is now the primary cluster, but the ZDM Proxy is still keeping Origin up-to-date via dual writes.
Verifying the read routing change
Once the read routing configuration change has been rolled out, you may want to verify that reads are correctly sent to Target as expected. This is not a required step, but you may wish to do it for peace of mind.
|
Issuing a The ZDM Proxy handles reads to system tables differently, by intercepting them and always routing them to Origin, in some cases partly populating them at proxy level. This means that system reads are not representative of how the ZDM Proxy routes regular user reads: even after you switched the configuration to read from Target as the primary cluster, all system reads will still go to Origin. Although |
Verifying that the correct routing is taking place is a slightly cumbersome operation, due to the fact that the purpose of the ZDM process is to align the clusters and therefore, by definition, the data will be identical on both sides.
For this reason, the only way to do a manual verification test is to force a discrepancy of some test data between the clusters. To do this, you could consider using the Themis sample client application. This client application connects directly to Origin, Target and the ZDM Proxy, inserts some test data in its own table and allows you to view the results of reads from each source. Please refer to its README for more information.
Alternatively, you could follow this manual procedure:
-
Create a small test table on both clusters, for example a simple key/value table (it could be in an existing keyspace, or in one that you create specifically for this test). For example
CREATE TABLE test_keyspace.test_table(k TEXT PRIMARY KEY, v TEXT);. -
Use
cqlshto connect directly to Origin. Insert a row with any key, and with a value specific to Origin, for exampleINSERT INTO test_keyspace.test_table(k, v) VALUES ('1', 'Hello from Origin!');. -
Now, use
cqlshto connect directly to Target. Insert a row with the same key as above, but with a value specific to Target, for exampleINSERT INTO test_keyspace.test_table(k, v) VALUES ('1', 'Hello from Target!');. -
Now, use
cqlshto connect to the ZDM Proxy (see here for how to do this) and issue a read request for this test table:SELECT * FROM test_keyspace.test_table WHERE k = '1';. The result will clearly show you where the read actually comes from.
