Adding an Amazon S3 backup location

Add an Amazon S3 backup location. Set a retention policy for the backup location.

Add an Amazon S3 backup location. For more details, see backing up to Amazon S3 and the Amazon S3 transfer acceleration documentation.

Prerequisites

  • Java 8 is required to store at an S3 location.
  • Make sure you have the proper AWS IAM privileges.
  • An S3 bucket destination must be unique and self-contained. Any defined destination cannot be contained within another backup destination. For example, if you set up a backup location to mybucket1, do not designate another backup location to mybucket1/myfolder1. If you set up a mybucket1/myfolder1 location, do not set up another location as mybucket1/myfolder1/mysubfolder1. Folders are supported; however, bucket paths cannot share any portion of a backup destination.
  • As a recommended best practice, limit an S3 bucket to a single keyspace for OpsCenter backups. Because every backup job gathers a list of all existing data files before the transfer to S3 can start, that process can take more time as the number of files grow in the bucket.
    Important: Moving backup files from Amazon S3 to Amazon Glacier is not supported by the OpsCenter Backup Service.
Important: The Backup Service requires control over the data and structure of its destination locations. The AWS S3 bucket and the Local file system destinations must be dedicated for use only by OpsCenter. Any additional directories or files in those destinations can prevent the Backup Service from properly conducting a Backup or Restore operation.
Warning: The AWS credentials and bucket names are stored in cluster_name.conf (with the exception of ad hoc backups). Be sure to use proper security precautions to ensure that this file is not readable by unauthorized users.

Procedure

  1. Access the Create (or Edit) Backup dialog:
    1. Click cluster name > Services.
    2. Click the Details link for the Backup Service.
    3. In the Activity tab, click Create Backup.
  2. In the Create or Edit Backup dialog, under Location, click +Add Location.
    The Add Location dialog appears.

    Add Location dialog S3 location with Retention Policy for scheduled backups

  3. Select Amazon S3 as the backup Location.
  4. Enter the S3 Bucket name.
    Note: The bucket name must be at least 4 characters long. Bucket names should only contain lowercase letters, numbers, and hyphens. Additionally, OpsCenter requires that bucket prefixes contain only lowercase letters, numbers, and safe characters. See the S3 guidelines for more details about bucket naming restrictions.
    Tip: To indicate a bucket subfolder location, delineate the bucket name from the folder name with a forward slash (/) character. Example: mybucket/myfolder/mysubfolder. Remember that slashes are not allowed within bucket or folder names themselves.
  5. Select the source type of your AWS credentials.
    Warning: The AWS credentials and bucket names are stored in cluster_name.conf (with the exception of ad hoc backups). Be sure to use proper security precautions to ensure that this file is not readable by unauthorized users.
    OptionDescription
    User-Supplied Credentials Enter your AWS key and AWS Secret. See AWS Access Keys.
    AWS Credential Provider chain Use the default credential provider chain to locate AWS credentials. See Working with AWS Credentials.
  6. Optional: Select any throttling, compressing, or encryption of the data:
    1. To avoid saturating your network, set a maximum upload rate. Select Throttle S3 transfer rate and set the maximum MB per second.
      Note: When the AWS CLI labs feature is enabled, the S3 throttle is ignored at this time. A tooltip also mentions this current limitation. See Tuning throttling when using AWS CLI.
    2. To compress the backup data, select Enable compression. Compression reduces the amount of data going through your network and reduces the disk and data usage but increases the CPU load for the server.
    3. To enable server-side S3 encryption, select Enable S3 server-side encryption. Enabling server-side encryption increases the security of your backup files, but increases the time it takes to complete a backup. For more information on S3 server-side encryption, see Using Server Side Encryption on the AWS website.

      Choose the type of encryption you want to use:

      Option Description
      256-bit Advanced Encryption Standard SSE-S3 encryption encrypts each file in the backup set with a unique key, including the key itself, using a 256-bit AES cypher.
      KMS Managed Encryption SSE-KMS encryption uses customer master keys (CMKs) to encrypt Amazon S3 objects.

      Enter a KMS Key ID that is associated with your AWS account.

    4. To backup nodes running in multiple regions to a single bucket, select Enable S3 transfer acceleration. Instead of traffic crossing over the internet, acceleration mode uses Amazon CloudFront to cache S3 requests. Because the CloudFront servers are closer to the nodes in each region, the backup latency is reduced.
      Note: Enabling S3 transfer acceleration can cause performance degradation, and might slow a standard backup configuration. Use this option only if backing up nodes in multiple regions to a single bucket.
  7. Optional: For scheduled backups, indicate how long the snapshot data should be retained by selecting a Retention Policy. Retain All (default) saves the snapshot data indefinitely. Or, define a set period of time. After the snapshot data is older than the time set in Retention Policy, the snapshot data is deleted.

    DataStax strongly recommends setting a retention policy to periodically remove backups. This practice helps to avoid long-term performance issues caused by an excessive number of backups.

    Note: Setting a Retention Policy is not available for an ad hoc (Run Now) backup.
  8. Click Save Location.
    The newly added S3 location displays in the Location pane of the Create or Edit Backup dialog.

    Backup locations

    Click the edit icon to the edit a location and its retention policy if applicable. Click the delete icon to delete a location. The On Server location cannot be deleted.

  9. Click Save Backup, or Create Backup as applicable.