Setting the NodeSync deadline

Each table with NodeSync enabled has a deadline_target_sec property. This is the target for the maximum time between 2 validations of the same data. As long as the deadline is met, all parts of the ring (for the table) are validated at least that often.

The deadline (deadline_target_sec) relates to grace period (gc_grace_seconds). The deadline should always be less than or equal to the grace period. As long as the deadline is met, no data is resurrected due to tombstone purging.

The deadline defaults to which ever is longer, the grace period or four days. Typically an acceptable default, unless the table has a grace period of zero. For testing, the deadline value can be less than the grace period. Verify for a few weeks if a lower gc_grace value is realistic without taking risk before changing it.

NodeSync prioritize segments in order to try to meet the deadline. The next segment to validate at any given time is the one the closest to missing its deadline. For example, if table 1 has half the deadline of table 2, table 1 validates approximately twice as often as table 2.

Use OpsCenter to get a graphical representation of the NodeSync validation status. See Viewing NodeSync Status.

The syntax to change the per-table nodesync property:

ALTER TABLE <table_name>
WITH nodesync = { 'enabled': 'true',
                  'deadline_target_sec': <value> };

If your set deadline is too aggressive or the NodeSync rate is too low, you may encounter a warning log such as:

WARN  [NodeSyncMaintenanceTasks:1] 2023-10-11 21:21:25,117 -
The configured NodeSync rate on this node (256kB/s) is too low to possibly validate all NodeSync
-enabled tables within their respective deadline ('deadline_target_sec' property). This can be
fixed by increasing the rate and/or increasing table deadlines. With the current deadlines and
current table size, the theoretical minimum rate would be 264kB/s, but we would recommend a
_minimum_ of 422kB/s and ideally 774kB/s to account for node failures, temporary slow nodes
and future data growth. Please check 'nodetool nodesyncservice ratesimulator' for more details
on how those values are computed.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000,