CREATE BACKUP STORE

Creates a new backup store.

Creates a new backup store. Backup stores can be created using different storage classes: local file storage, Amazon S3 storage, or Google Cloud storage. Depending upon the storage class, different collections of settings are required.

Synopsis

CREATE BACKUP STORE store_name
    USING 'store_class_name'
    WITH SETTINGS = settings ;
Table 1. Legend
Syntax conventions Description
UPPERCASE Literal keyword.
Lowercase Not literal.
Italics Variable value. Replace with a user-defined value.
[] Optional. Square brackets ( [] ) surround optional command arguments. Do not type the square brackets.
( ) Group. Parentheses ( ( ) ) identify a group to choose from. Do not type the parentheses.
| Or. A vertical bar ( | ) separates alternative elements. Type any one of the elements. Do not type the vertical bar.
... Repeatable. An ellipsis ( ... ) indicates that you can repeat the syntax element as often as required.
'Literal string' Single quotation ( ' ) marks must surround literal strings in CQL statements. Use single quotation marks to preserve upper case.
{ key : value } Map collection. Braces ( { } ) enclose map collections or key value pairs. A colon separates the key and the value.
<datatype1,datatype2> Set, list, map, or tuple. Angle brackets ( < > ) enclose data types in a set, list, map, or tuple. Separate the data types with a comma.
cql_statement; End CQL statement. A semicolon ( ; ) terminates all CQL statements.
[--] Separate the command line options from the command arguments with two hyphens ( -- ). This syntax is useful when arguments might be mistaken for command line options.
' <schema> ... </schema> ' Search CQL only: Single quotation marks ( ' ) surround an entire XML schema declaration.
@xml_entity='xml_entity_type' Search CQL only: Identify the entity and literal value to overwrite the XML element in the schema and solrConfig files.
store_name

Name of the backup store. Store names can use alphanumeric characters and underscores only. They are case insensitive.

store_class_name
Name of the backup store class for the backup store. Available store classes:
  • FSBlobStore: Local (node) or NFS file system storage.
  • S3BlobStore: Amazon Web Services (AWS) S3 or compatible storage.
  • AzureBlobStore: Microsoft Azure storage.
  • GoogleCloudBlobStore: Google cloud storage.
  • CompositeBlobStore: A store comprising multiple backup store types used to back up separate datacenters to separate backup stores.
settings
A key/value list of settings particular to the store class. For example, for an FSBlobStore class:
settings = {'path':'/path/to/data', 'retention_time':'1w'}

Common settings for all store classes

The following settings can be used with any store class.

Setting Description Type Required
transfer_rate_limit_mbs Maximum upload rate per node in MB per second. int false
retention_time Time duration for which a snapshot is retained. duration false
retention_number Number of snapshots retained. int false

The parameters retention_time and retention_number can be used individually or in combination. Setting either to 0 ignores the respective parameter.

Important: The transfer_rate_limit_mbs setting is not supported for Composite backup stores.

Examples

Retain backup younger than one week:

'retention_time':'1w'
Retain a minimum of 3 full backup at any point in time:
'retention_number':3
Note: Any non-full backup with a status of INCOMPLETE or QUORUM that occurs between the full snapshots is retained as well.
Retain a mininum of 3 full backups even if older than one week:
'retention_time':'1w','retention_number':3 
Important: If, for example, there happen to be four full backups within the retention_time window, all four are retained. If there are only two backups, however, the last backup older than the retention_time window will also be retained. In addition, any non-full backup with a status of INCOMPLETE or QUORUM that occurs between the full backups is retained as well.

FSBlobStore class settings

The following settings are applicable only to the FSBlobStore class.

Setting Description Type Required
path Path where snapshots are stored. The directory must exist. text true

AzureBlobStore class settings

The following settings are applicable only to the AzureBlobStore class.

Setting Description Type Required
container Azure container in which snapshots are stored. text true
endpoint Azure endpoint. May be replaced with the Azure account name. text true
sas_token The Azure sas_token. text true unless Azure Managed Identity authentication mechanism is in use.

GoogleCloudBlobStore class settings

The following settings are applicable only to the GoogleCloudBlobStore class.

Setting Description Type Required
bucket Google Cloud bucket in which snaphots are stored. text true
client_email Email address associated with the Google Cloud account. text false unless private_key is set
client_id Client ID associated with the Google Cloud account. text false
private_key Private key associated with the Google Cloud account. text false unless client_email is set
private_key_id Private key ID of the private_key associated with the Google Cloud account. text false
project_id Project ID associated with the Google Cloud bucket. text false
endpoint Google Cloud Endpoint. See https://cloud.google.com/endpoints/docs/quickstart-endpoints. text false

Authentication details

If no credentials are set in the GoogleCloudBlobStore settings, Application Default Credentials (ADC) are used. Credentials are checked in the following order:
  1. ADC checks the environment variable, GOOGLE_APPLICATION_CREDENTIALS, and, if set, uses the referenced service account file.
  2. If GOOGLE_APPLICATION_CREDENTIALS is not set, ADC uses the default service account provided by the Compute Engine, Kubernetes Engine, App Engine, or Cloud Functions depending on the environment on which the application is running.
  3. If no credentials are found, an error occurs.

See: https://cloud.google.com/docs/authentication/production.

S3BlobStore class settings

The following settings are applicable to only the S3BlobStore class.

Setting Description Type Required
bucket AWS bucket in which snapshots are stored. text true
region AWS region, an alternative to the endpoint setting. text true if endpoint is not set
access_key Access key used for authentication. text false
secret_key Secret key used for authentication. text false
default_socket_timeout Socket timeout in ms.

Default: 50,000

int false
max_error_retry Maximum retries when an error occurs.

Default: 3

int false
default_throttle_retries Enables retry throttling.

Default: true

int false
server_side_encryption Enable server-side encryption (AES256).

Default: true

boolean false
accelerated_mode_enabled Enable AWS transfer acceleration. Only applicable to AWS S3. See https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html.

Default: true

boolean false
storage_class Storage class in which Binary Large Objects (BLOBS) are stored. See https://aws.amazon.com/s3/storage-classes/. text false
canned_acl Access Control List (ACL) for buckets and objects. text false
endpoint Endpoint for the AWS S3 server, an alternative to the region setting. text true if region is not set.

Authentication details

If no details are specified in the S3BlobStore settings, the DefaultAWSCredentialsProviderChain is used. The DefaultAWSCredentialsProviderChain searches for credentials in the following order:
  1. The environment variables, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
  2. The Java system properties, aws.accessKeyId and aws.secretKey.
  3. A credential profile located at ~/.aws/credentials.
  4. Credentials provided through the Amazon EC2 container service if the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable is set and the security manager has permission to access that variable.
  5. Instance profile credentials delivered using the Amazon EC2 metadata service.

CompositeBlobStore class settings

The following setting is applicable to only the CompositeBlobStore class.

Setting Description Type Required
data-center-name The key must be a valid DSE datacenter name. text The value must be an existing configured backup store:
  • FSBlobStore
  • AzureBlobStore
  • GoogleCloudBlobStore
  • S3BlobStore

Examples

Create a local file system backup store

Create a local file system backup store with defaults:
CREATE BACKUP STORE fs_store_1
USING 'FSBlobStore' WITH settings = {'path':'/my/local/path'};

Create a Google backup store

Create a Google backup store:
CREATE BACKUP STORE google_store
USING 'GoogleBlobStore' WITH settings = {'bucket': 'my_google_bucket';

Create an Amazon S3 backup store

Create an Amazon S3 backup store with a custom retention period and number of backups:
CREATE BACKUP STORE s3_store
USING 'S3BlobStore' WITH settings = {'bucket': 'my_s3_bucket', 
                                     'region': 'us-west-1',
                                     'retention_time':'1w'
                                     'retention_number' : 4};
Important: For more information on retention policies, see Customize backup retention policy.

Create a Composite backup store

Create a Composite backup store:
CREATE BACKUP STORE composite_store
USING 'CompositeBlobStore' WITH settings = {'DC1': 'fs_store_1', 'DC2': 's3_store'};