Backup Service configuration options reference

Configuration files

Filename

Location dependent on the type of installation

address.yaml

  • Package installations: /var/lib/datastax-agent/conf/address.yaml

  • Tarball installations: install_location/conf/address.yaml

opscenterd.conf

  • Package installations: /etc/opscenter/opscenterd.conf

  • Tarball installations: install_location/conf/opscenterd.conf

cluster_name.conf

  • Package installations: /etc/opscenter/clusters/cluster_name.conf

  • Tarball installations: install_location/conf/clusters/cluster_name.conf

The Backup Service configuration options reference lists the available configuration options in alphabetical order for each configuration file. See Configuration files for OpsCenter more information about configuration files and their precedence behavior.

OpsCenter daemon configuration options for the Backup Service

The following options are available in opscenterd.conf:

  • [backup_service] auto_sync_retry_attempts

    The number of times to auto retry sending a backup to a destination if there are failures. Default: 3.

  • [backup_service] auto_sync_retry_delay

    The amount of time in minutes to wait between retry attempts. Default: 60.

  • [agents] agent_tmp_dir

    Set the path to a temporary directory used by the DataStax agents to hold files for backup and repair service.

Cluster configuration options for the Backup Service

The following options are available in cluster_name.conf:

  • [agents] backup_staging_dir

    This path specifies the directory where commit logs archived to by cassandra. From there the commit logs are sent to the configured destinations and the backup_storage_dir.

  • [agents] concurrent_snapshot_list_requests

    The number of concurrent get snapshot info requests. The default value is 1.

  • [agents] remote_backup_region

    The S3 region region to connect to for remote backup/restore. The default value is us-west-1.

  • [agents] restore_req_update_period

    The frequency in seconds that a restore reports progress back to OpsCenter. The default value is automatically calculated based on cluster size. To optimize performance, larger clusters have a longer threshold in which restore progress is sent to the UI. Configure this property accordingly for more or less frequent updates during a restore.

Lower numbers might impact performance during a restore.

  • [agents] snapshot_wait

    See [cassandra] snapshot_threshold

  • [backups] failure_threshold

    The percentage of the cluster that can fail to respond before a remote destination restore action fails. Default: 50.

  • [backups] restore_init_throttle

    The number of DataStax Agents on which OpsCenter will concurrently start the restore process. The default value is 20.

  • [backups] restore_sleep

    How long OpsCenter will sleep between batches of starting the restore process, set in restore_init_throttle. The default value is 5 seconds.

  • [cassandra] snapshot_threshold

    The number of nodes in the cluster before OpsCenter will switch from running a backup immediately upon receiving a request to scheduling the backup to run after the next full minute, plus any time set in snapshot_wait. The default value is 10. If there are less than 10 nodes in the cluster, then OpsCenter will tell all nodes to take a snapshot as soon as it is requested. If there are more than 10 nodes, OpsCenter will tell all nodes to take a snapshot at the current time rounded to the next minute, plus snapshot_wait seconds.

Agent configuration options for the Backup Service

The following agent configuration options are available in address.yaml:

  • backup_staging_dir

    The directory where commitlogs are copied after they are written to disk from DSE. The DataStax Agents monitor this directory and move commitlogs to the configured destinations. After all destinations receive the relevant commit logs, the logs are moved to the backup_storage_dir. The default location is /var/lib/datastax-agent/commitlogs/. Example: backup_staging_dir: /var/lib/datastax-agent/commitlogs/

  • backup_storage_dir

    The directory where On Server commitlog backups are stored after being copied to all configured destinations. The directory will be cleaned based on a configured retention policy for an On Server location. The directory should be large enough to hold commitlogs for the length of the retention policy. The default location is /var/lib/datastax-agent/backups/. Example: backup_storage_dir: /var/lib/datastax-agent/backups/

  • destination_pretest_timeout

    The maximum amount of time in seconds to verify a destination can be written to and read from. Default: 60. Example: destination_pretest_timeout: 60

  • max_file_transfer_attempts

    The maximum number of attempts to upload a file or create a remote destination. Default: 3. Example: max_file_transfer_attempts: 30

  • remote_backup_region

    The AWS region to use for remote backup transfers. Default: us-west-1. Example: remote_backup_region: us-west-1

  • remote_backup_retries

    The number of attempts to make when file download fails during a restore. Default: 3. Example: remote_backup_retries: 3

  • remote_backup_timeout

    The timeout in milliseconds for the connection used to push backups to remote destinations. Default: 1000. Example: remote_backup_timeout: 1000

  • remote_verify_initial_delay

    Initial delay in milliseconds to wait before checking if a file was successfully uploaded during a backup operation. This configuration option works in conjunction with the remote_verify_max option to distinguish between broken versus tardy backups when cleaning up SSTables. The remote_verify_initial_delay value doubles each time a file transfer validation failure occurs until the value exceeds the remote_verify_max value. Default: 1000 (1 second). Example: remote_verify_initial_delay: 1000

  • remote_verify_max

    The maximum time period to wait after a file upload completed but is still unreadable from the remote destination. When this delay is exceeded, the transfer is considered failed. This configuration option works in conjunction with the remote_verify_initial_delay option to distinguish between broken versus tardy backups when cleaning up SSTables. Default: 30000 (30 seconds). Example: remote_verify_max: 300000

  • restore_on_transfer_failure

    When set to true, a failed file transfer from the remote destination will not halt the restore process. process. A future restore attempt uses any successfully transferred files. Default: false. Example: restore_on_transfer_failure: false

  • restore_req_update_period

    The frequency in seconds with which status updates are sent to opscenterd during Restore operations in the Backup Service. Default: 60. Example: restore_req_update_period: 60

  • sstableloader_max_heap_size

    The maximum heap size used by the sstableloader during restore operations. Default: 256M. Example: sstableloader_max_heap_size: 256M

  • tmp_dir

    The directory used to temporarily stage files when restoring. The default location is /var/lib/datastax-agent/tmp. Example: tmp_dir: /var/lib/datastax-agent/tmp/

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2024 DataStax | Privacy policy | Terms of use

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com