CREATE BACKUP STORE

Creates a new backup store.

Creates a new backup store. Backup stores can be created using different storage classes: local file storage, Amazon S3 storage, or Google Cloud storage. Depending upon the storage class, different collections of settings are required.

Synopsis

CREATE BACKUP STORE store_name
    USING 'store_class_name'
    WITH SETTINGS = settings ;

Table 1. Legend
Syntax conventions	Description
UPPERCASE	Literal keyword.
Lowercase	Not literal.
`Italics`	Variable value. Replace with a user-defined value.
`[]`	Optional. Square brackets ( `[]` ) surround optional command arguments. Do not type the square brackets.
`( )`	Group. Parentheses ( `( )` ) identify a group to choose from. Do not type the parentheses.
`\|`	Or. A vertical bar ( `\|` ) separates alternative elements. Type any one of the elements. Do not type the vertical bar.
`...`	Repeatable. An ellipsis ( `...` ) indicates that you can repeat the syntax element as often as required.
`'Literal string'`	Single quotation ( `'` ) marks must surround literal strings in CQL statements. Use single quotation marks to preserve upper case.
`{ key : value }`	Map collection. Braces ( `{ }` ) enclose map collections or key value pairs. A colon separates the key and the value.
`<datatype1,datatype2>`	Set, list, map, or tuple. Angle brackets ( `< >` ) enclose data types in a set, list, map, or tuple. Separate the data types with a comma.
`cql_statement;`	End CQL statement. A semicolon ( `;` ) terminates all CQL statements.
`[--]`	Separate the command line options from the command arguments with two hyphens ( `--` ). This syntax is useful when arguments might be mistaken for command line options.
`' <schema> ... </schema> '`	Search CQL only: Single quotation marks ( `'` ) surround an entire XML schema declaration.
`@xml_entity='xml_entity_type'`	Search CQL only: Identify the entity and literal value to overwrite the XML element in the schema and solrConfig files.

store_name

Name of the backup store. Store names can use alphanumeric characters and underscores only. They are case insensitive.

store_class_name

Name of the backup store class for the backup store. Available store classes:

FSBlobStore: Local (node) or NFS file system storage.
S3BlobStore: Amazon Web Services (AWS) S3 or compatible storage.
AzureBlobStore: Microsoft Azure storage.
GoogleCloudBlobStore: Google cloud storage.
CompositeBlobStore: A store comprising multiple backup store types used to back up separate datacenters to separate backup stores.

settings

A key/value list of settings particular to the store class. For example, for an FSBlobStore class:

settings = {'path':'/path/to/data', 'retention_time':'1w'}

Common settings for all store classes

The following settings can be used with any store class.


Setting	Description	Type	Required
`transfer_rate_limit_mbs`	Maximum upload rate per node in MB per second.	int	false
`retention_time`	Time duration for which a snapshot is retained.	duration	false
`retention_number`	Number of snapshots retained.	int	false

The parameters retention_time and retention_number can be used individually or in combination. Setting either to 0 ignores the respective parameter.

Important: The transfer_rate_limit_mbs setting is not supported for Composite backup stores.

Examples

Retain backup younger than one week:

'retention_time':'1w'

Retain a minimum of 3 full backup at any point in time:

'retention_number':3

Note: Any non-full backup with a status of INCOMPLETE or QUORUM that occurs between the full snapshots is retained as well.

Retain a mininum of 3 full backups even if older than one week:

'retention_time':'1w','retention_number':3

Important: If, for example, there happen to be four full backups within the retention_time window, all four are retained. If there are only two backups, however, the last backup older than the retention_time window will also be retained. In addition, any non-full backup with a status of INCOMPLETE or QUORUM that occurs between the full backups is retained as well.

FSBlobStore class settings

The following settings are applicable only to the FSBlobStore class.


Setting	Description	Type	Required
`path`	Path where snapshots are stored. The directory must exist.	text	true

AzureBlobStore class settings

The following settings are applicable only to the AzureBlobStore class.


Setting	Description	Type	Required
`container`	Azure container in which snapshots are stored.	text	true
`endpoint`	Azure endpoint. May be replaced with the Azure account name.	text	true
`sas_token`	The Azure sas_token.	text	true unless Azure Managed Identity authentication mechanism is in use.

GoogleCloudBlobStore class settings

The following settings are applicable only to the GoogleCloudBlobStore class.


Setting	Description	Type	Required
`bucket`	Google Cloud bucket in which snaphots are stored.	text	true
`client_email`	Email address associated with the Google Cloud account.	text	false unless `private_key` is set
`client_id`	Client ID associated with the Google Cloud account.	text	false
`private_key`	Private key associated with the Google Cloud account.	text	false unless `client_email` is set
`private_key_id`	Private key ID of the `private_key` associated with the Google Cloud account.	text	false
`project_id`	Project ID associated with the Google Cloud bucket.	text	false
`endpoint`	Google Cloud Endpoint. See https://cloud.google.com/endpoints/docs/quickstart-endpoints.	text	false

Authentication details

If no credentials are set in the GoogleCloudBlobStore settings, Application Default Credentials (ADC) are used. Credentials are checked in the following order:

ADC checks the environment variable, GOOGLE_APPLICATION_CREDENTIALS, and, if set, uses the referenced service account file.
If GOOGLE_APPLICATION_CREDENTIALS is not set, ADC uses the default service account provided by the Compute Engine, Kubernetes Engine, App Engine, or Cloud Functions depending on the environment on which the application is running.
If no credentials are found, an error occurs.

See: https://cloud.google.com/docs/authentication/production.

S3BlobStore class settings

The following settings are applicable to only the S3BlobStore class.


Setting	Description	Type	Required
`bucket`	AWS bucket in which snapshots are stored.	text	true
`region`	AWS region, an alternative to the `endpoint` setting.	text	true if `endpoint` is not set
`access_key`	Access key used for authentication.	text	false
`secret_key`	Secret key used for authentication.	text	false
`default_socket_timeout`	Socket timeout in ms. Default: 50,000	int	false
`max_error_retry`	Maximum retries when an error occurs. Default: 3	int	false
`default_throttle_retries`	Enables retry throttling. Default: true	int	false
`server_side_encryption`	Enable server-side encryption (AES256). Default: true	boolean	false
`accelerated_mode_enabled`	Enable AWS transfer acceleration. Only applicable to AWS S3. See https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html. Default: true	boolean	false
`storage_class`	Storage class in which Binary Large Objects (BLOBS) are stored. See https://aws.amazon.com/s3/storage-classes/.	text	false
`canned_acl`	Access Control List (ACL) for buckets and objects.	text	false
`endpoint`	Endpoint for the AWS S3 server, an alternative to the `region` setting.	text	true if `region` is not set.

Authentication details

If no details are specified in the S3BlobStore settings, the DefaultAWSCredentialsProviderChain is used. The DefaultAWSCredentialsProviderChain searches for credentials in the following order:

The environment variables, AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
The Java system properties, aws.accessKeyId and aws.secretKey.
A credential profile located at ~/.aws/credentials.
Credentials provided through the Amazon EC2 container service if the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable is set and the security manager has permission to access that variable.
Instance profile credentials delivered using the Amazon EC2 metadata service.

CompositeBlobStore class settings

The following setting is applicable to only the CompositeBlobStore class.


Setting	Description	Type	Required
`data-center-name`	The key must be a valid DSE datacenter name.	text	The value must be an existing configured backup store: `FSBlobStore` `AzureBlobStore` `GoogleCloudBlobStore` `S3BlobStore`

Examples

Create a local file system backup store

Create a local file system backup store with defaults:

CREATE BACKUP STORE fs_store_1
USING 'FSBlobStore' WITH settings = {'path':'/my/local/path'};

Create a Google backup store

Create a Google backup store:

CREATE BACKUP STORE google_store
USING 'GoogleBlobStore' WITH settings = {'bucket': 'my_google_bucket';

Create an Amazon S3 backup store

Create an Amazon S3 backup store with a custom retention period and number of backups:

CREATE BACKUP STORE s3_store
USING 'S3BlobStore' WITH settings = {'bucket': 'my_s3_bucket', 
                                     'region': 'us-west-1',
                                     'retention_time':'1w'
                                     'retention_number' : 4};

Important: For more information on retention policies, see Customize backup retention policy.

Create a Composite backup store

Create a Composite backup store:

CREATE BACKUP STORE composite_store
USING 'CompositeBlobStore' WITH settings = {'DC1': 'fs_store_1', 'DC2': 's3_store'};