Use the NodeSync Service

NodeSync automatically and continuously synchronizes replicas of designated keyspaces and tables of a cluster as a background process. The DSE OpsCenter NodeSync Service is initially enabled by default, but keyspaces and tables must be opted in to the NodeSync Service.

NodeSync Service, Repair Service, and limitations on repair operations

Unlike incremental repairs in the Repair Service, the NodeSync Service does not build a Merkle tree for comparisons and stream differences for compaction. Rather, the NodeSync incrementally scans a token range, compares data a page at a time, and sends data over the standard write path.

The nodetool repair command and repair operations run from the OpsCenter node administration menu fail when targeting a NodeSync-enabled table. Tables with NodeSync enabled are skipped for nodetool repair operations run at the keyspace level.

The legacy Repair Service can work in conjunction with the NodeSync Service.

Tables without NodeSync can use the legacy Repair Service freely.

If preferable for an environment, tables can be repaired (synchronized) using either the legacy Repair Service or the NodeSync Service. The Repair Service does not start any subrange or incremental repairs on tables that have NodeSync enabled and running until the NodeSync operation completes. If a legacy repair (also known as classic incremental or subrange repairs) is in progress for a table that has NodeSync enabled, NodeSync waits for the classic repair to finish before proceeding. If a classic repair is launched against a keyspace with NodeSync enabled on some but not all tables, a warning is logged and classic repair jobs are started for the non-NodeSync tables.

For operational simplicity and better performance, DataStax recommends running NodeSync Service exclusively rather than the legacy Repair Service.

Prepare your DSE cluster for NodeSync

  • NodeSync is enabled in DSE by default. If you disabled NodeSync in DSE, you must enable NodeSync to use the NodeSync Service.

    Incremental NodeSync is available in DataStax Enterprise (DSE) 6.8 and later.

    1. If any keyspaces are empty, create at least one table in each keyspace that you want to opt in to NodeSync.

      You cannot enable NodeSync on empty keyspaces in OpsCenter.

  • If OpsCenter authentication is enabled, set user permissions for the NodeSync Service for the applicable roles and clusters.

Enable the NodeSync Service for keyspaces and tables

NodeSync and incremental NodeSync are enabled by default on a cluster, but you must opt in keyspaces and tables for synchronization:

  1. Click the name of the cluster that you want to manage, and then click Services.

  2. Select Details for the NodeSync Service.

  3. Select the Settings tab.

    NodeSync Settings shows only keyspaces and tables eligible for NodeSync. Some tables are not shown, and they are exempt from NodeSync because the keyspace has a replication factor of 1, or it is a reserved system keyspace. Select View excluded tables to view the excluded keyspaces and tables.

  4. Select the keyspaces and tables to configure NodeSync for your environment, and then select Configure.

    By default, you can configure up to 50 tables at once. If you encounter schema agreement errors when enabling or disabling multiple tables, increase the value of max_request_tables.

  5. Enable NodeSync or incremental NodeSync for the selected keyspaces and tables.

    Enabling NodeSync sends an ALTER TABLE statement that is effective immediately; You don’t need to restart DSE.

After you enable NodeSync, monitor NodeSync operations for keyspace and tables:

Monitor NodeSync

There are several ways to monitor NodeSync operations.

NodeSync metrics

NodeSync metrics are available to indicate progress on repaired data and objects (rows and range tombstones), validated data and objects, and processed pages details. Set up alerts and dashboard graphs based on NodeSync metrics pertinent to your production environment.

NodeSync rule for the Best Practice Service

The NodeSync Not Running rule fails if the NodeSync Service is not running on a node, unless the rule has been turned off in the Best Practice Service.

NodeSync Service Best Practice Rule alerts when NodeSync is not running

NodeSync status and details

View the progress and status of NodeSync operations for keyspace and tables in OpsCenter Monitoring. The NodeSync Status page provides summary and detailed health status information.

  1. Click the name of the cluster you want to inspect, and then click Services.

  2. Select Details for the NodeSync Service.

    The Status tab shows the current progress of NodeSync operations, including the time of the latest data refresh.

    Each cluster, keyspace, and table has a color-coded legend that conveys its NodeSync status (listed from best to worst) when hovering over a particular progress bar:

    • Green: Percentage of the data was synced within the first 50% of the time period for the NodeSync deadline.

    • Yellow: Percentage of the data was synced within the first 90% of the time period for the NodeSync deadline.

    • Orange: Percentage of the data was synced within the last 10% of the time period for the NodeSync deadline.

    • Red: Percentage of the data was not synced before the NodeSync deadline. DataStax recommends that you increase your NodeSync throughput.

    The NodeSync deadline is a table parameter that specifies the maximum number of seconds to validate all segments of the table. If the deadline is not explicitly set for a table, the deadline defaults to whichever is greater: gc_grace_seconds or the minimum deadline target of 4 days. You can set the deadline value with the deadline_target_sec parameter in an ALTER TABLE statement. For example:

    ALTER TABLE table_name WITH nodesync={ 'enabled': 'true', 'deadline_target_sec' : <some value>}

    Select a progress bar section to view a tooltip with more details about progress status. The Services panel at Cluster > Nodes also shows the NodeSync progress bar.

  3. Select the Keyspaces or Tables link to view the Detailed Health Status.

  4. Select a table within Detailed Health Status to view NodeSync Details for that table.

Configure the NodeSync data refresh interval

The NodeSync refresh_period is the frequency, in seconds, at which NodeSync status data is refreshed in the NodeSync metrics table in OpsCenter Monitoring. The default is 300 seconds (5 minutes). The interval is measured from the end of the last refresh operation. For example, If a status refresh takes 2 minutes to complete, the refresh period timer starts again and refreshes NodeSync Status after another 5 minutes.

To modify the refresh_period, do the following:

  1. Edit the relevant configuration file for the configuration scope and your installation type:

    File Scope Package installation location Tarball installation location

    opscenterd.conf

    Global, all clusters

    /etc/opscenter/opscenterd.conf

    INSTALL_DIRECTORY/conf/opscenterd.conf

    CLUSTER_NAME.conf

    Cluster-specific

    /etc/opscenter/clusters/

    INSTALL_DIRECTORY/conf/clusters/

    address.yaml

    Node-specific

    /var/lib/datastax-agent/conf/address.yaml

    INSTALL_DIRECTORY/conf/address.yaml

  2. In the [nodesync] section, set the refresh_period option.

    The following example sets the interval to 15 minutes (900 seconds):

    [nodesync]
    refresh_period=900
  3. Restart opscenterd.

  4. If you edited address.yaml, restart each DataStax Agent.

Tune throughput for NodeSync

NodeSync is not self-tuning.

If NodeSync operations don’t finish before the expiration of a table’s GC grace period, you must adjust NodeSync throughput with the rate_in_kb parameter. You can set this value in a Lifecycle Manager (LCM) Config Profile, and then deploy the configuration update by running a configuration job, as explained in the following steps. Alternatively, you can manually configure NodeSync with nodetool nodesyncservice.

Configure the NodeSync rate_in_kb for DSE using LCM:

  1. Select Config Profiles from the LCM navigation menu.

  2. Select the edit icon for the configuration profile to edit, or click Add config profile if you have not already created a profile.

  3. In the Config Profile pane under the Cassandra section, click cassandra.yaml.

  4. Scroll down to the NodeSync pane.

    Configure NodeSync rate in cassandra.yaml config profile of LCM
  5. Edit a value for rate_in_kb.

    The default is 1024 KB.

    To determine the optimal rate, use nodetool nodesyncservice ratesimulator. For more information, see Setting the NodeSync rate.

  6. Select Save to save the configuration profile.

  7. Go to the Clusters workspace in LCM, and then select the configuration profile to apply at the cluster, datacenter, or node level.

  8. Run a configuration job to push the updated configuration to all applicable nodes.

  9. View NodeSync Status in OpsCenter Monitoring to ensure that the new rate allows NodeSync jobs to complete within the required time frame.

Was this helpful?

Give Feedback

How can we improve the documentation?

© Copyright IBM Corporation 2026 | Privacy policy | Terms of use |  Manage Privacy Choices

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: Contact IBM