Adjusting or disabling the throttle for subrange repairs

Adjust or disable the throttle for the subrange repair cycle. The Repair Service automatically throttles repairs that it anticipates would complete quickly, dynamically spreading out the repair to alleviate server load and avoid repeating repairs unnecessarily within the specified time to completion window.

Adjust or disable the throttle for the subrange repair cycle.
Note: This feature is only applicable to subrange repairs. It has no impact on incremental repairs.

The Repair Service automatically throttles subrange repairs when the current repair cycle is estimated to finish significantly before the deadline specified by the time to completion. The throttle sleeps longer between repairs to alleviate load on the server during the repair process, dynamically tuning a repair job to run within the allotted time. Without the throttle enabled, repair processes that complete quickly unnecessarily repeats the repair. For instance, if it takes 3 days to repair a cluster and Repair Service is configured to complete in 9 days, a throttled Repair Service spreads out the repair over the 9 day period. In contrast, an unthrottled service would run 3 repair cycles in that same 9 day period.

The time_to_completion_target_percentage configuration option controls the frequency and pace of the subrange repair process. The throttle slows repairs or reduces parallel repair processes as necessary to prevent overloading the cluster while still completing a repair cycle within the specified time window designated by the Time to completion value. The default value for the target percentage to complete a repair is 65%.

Adjusting this setting is not typically required. The throttle is particularly useful for smaller clusters that have a subrange repair that completes within a short period of time, such as one day.

Disable the throttle only if you do not have any concerns about server workload impact from running repairs with greater frequency, or if you intend to configure advanced options such as max_parallel_repairs or min_repair_time.

cluster_name.conf

The location of the cluster_name.conf file depends on the type of installation:
  • Package installations: /etc/opscenter/clusters/cluster_name.conf
  • Tarball installations: install_location/conf/clusters/cluster_name.conf

opscenterd.conf

The location of the opscenterd.conf file depends on the type of installation:
  • Package installations: /etc/opscenter/opscenterd.conf
  • Tarball installations: install_location/conf/opscenterd.conf

Procedure

  1. Open for editing opscenterd.conf for all clusters, or cluster_name.conf for a specific cluster.
  2. Adjust the configuration as appropriate for your environment:
    Setting the percentage to zero disables the throttle:
    [repair_service]
time_to_completion_target_percentage=0

    Disabling the throttle is not recommended unless you intend to configure options such as maximum_parallel_repairs manually. See Setting the maximum for parallel subrange repairs.

    After disabling the throttle, the Repair Service logs messages that inform you about the behavior change.

    Setting the percentage to 100 spaces out subrange repairs over the entire time to completion window at the potential risk of not completing the repairs on time:
    [repair_service]
time_to_completion_target_percentage=100

    At 100% targeted percentage, the workload is spread out over a longer period of time, which lowers the impact on the cluster at any given time. However, missing the time to completion deadline is more likely without the extra time buffer afforded by the target default of 65%.

  3. Restart opscenterd.
  4. Monitor the subrange repair progress on the Status tab.
  5. Review the repair service log messages for awareness about the throttle recalibration impact on your environment.