Restoring from a backup

Restore data from a previously completed backup run from OpsCenter.

Restore data from any local or Amazon S3 backups that have been run by OpsCenter. You cannot use the OpsCenter Backup Service to restore from snapshots run with nodetool. You can pick any subset of tables that exist in the snapshot to restore.

Note: If the backup contains encrypted tables created prior to DataStax Enterprise 4.0.4 or 4.5.2, you will not be able to restore the snapshot. Due to a bug in Cassandra, backups containing encrypted table data from versions prior to 4.0.4 and 4.5.2 do not contain the necessary keys to restore the backup.

Prerequisites

  • To restore an encrypted backup, the agent must be granted password-less sudo access on the DataStax Enterprise nodes. This has already been granted if you used OpsCenter to install the agents. If you are running the agent as a different user than DataStax Enterprise and need to restore encrypted tables, you must manually restore the system_key table.
  • The Restore feature of the Backup Service leverages the sstableloader utility, which currently requires enabling the thrift server on all nodes before restoring. Before restoring, ensure the thrift server is enabled on all nodes.
  • When restoring tables that are Solr cores, if the table does not already exist, it will be automatically re-created as a CQL table. If you require this to be a thrift-based table, manually recreate the table prior to restoring. If you are restoring data from a thrift table that no longer exists, you are responsible for creating the table prior to restoring.
Important: The Backup Service requires control over the data and structure of its destination locations. The AWS S3 bucket and the Local file system destinations must be dedicated for use only by OpsCenter. Any additional directories or files in those destinations can prevent the Backup Service from properly conducting a Backup or Restore operation.

Procedure

  1. Click cluster name > Services.

  3. Click the Configure link for the Backup Service.
  4. In the Activity tab, click Restore Backup.

    The Restore from Backup dialog appears.

  5. Select the backup to restore in the list of backups and click Next.
    1. The Backups tab lists the available keyspace backups, including both scheduled and manual backups.
    2. If you have enabled commitlog backups, you can restore from a particular point-in-time by selecting the Point In Time tab. For more details, see Restoring a backup to a specific point-in-time.
    3. If you are restoring from an S3 location that is not listed in the Backups tab, select Other Location.

      Selecting a backup from Other Location is most commonly used when cloning a cluster, but can be used when this OpsCenter instance is not aware of the backup location.

      Enter the name of the bucket under S3 Bucket, then the AWS key and secret.

  6. Select the tables included in the backup you want to restore. Click the keyspace name to include all the tables in the keyspace. Click All Keyspaces to restore all the keyspaces.

    To select only specific tables, expand the keyspace name and select the tables.

    Note: Keep the following caveats in mind when creating and restoring backups:
    • Restoring a snapshot that contains only the system keyspace is not allowed. There must be both system and non-system keyspaces, or only non-system keyspaces in the snapshot you want to restore.
    • Restoring a snapshot that does not contain a table definition is not allowed.
    • Restoring a snapshot to a location with insufficient disk space fails. The Restore Report indicates which nodes do not have sufficient space and how much space is necessary for a successful restore. For more information and tips for preventative measures, see Monitoring sufficient disk space for restoring backups.


  7. Under Location, select the target cluster for the restored data.
    • The Location list is only available when both clusters are managed by the same instance of OpsCenter.
    • If you select a different cluster than the one that was backed up, the data will be cloned to the selected cluster.
    Note:

    Restoring encrypted tables to a different cluster will not work unless the encryption keys are identical, which is typically not the case.

  8. To remove the existing keyspace data before the data is restored, select Truncate/delete existing data before restore. This completely removes any updated data in the cluster for the keyspaces you are restoring.
  9. To prevent overloading the network, set a maximum transfer rate for the restore. Select Throttle stream throughput at ____ MB and set the maximum MB per second.
  10. Change the staging directory if necessary by setting the backup_staging_directory configuration option in address.yaml.
  11. Click Restore Backup.
  12. Click Start Restore to confirm when prompted.
    If the pre-restore checks detected schema differences that could not automatically be validated, the Restore Schema Validation dialog appears. Possible issues are listed and a comparison of the backup and current schema are presented side-by-side.

  13. Review the information to determine what adjustments if any need to be made to the current schema:
    • To rectify the schema issues and try the restore again afterward, click Cancel.
    • To proceed despite the schema mismatch, click Continue Restore.
      Warning: Attempting to restore a backup with an incompatible schema might result in corrupt or inaccessible data. Before forcing the restore, you might want to back up your current data.

Results

After the restore starts, the Restore Report dialog displays detailed information about the progress and status of the restore. The Restore Report dialog can be closed at any time without impacting the restore process. Reopen the report by clicking on the In Progress restore in the Activity tab. View the Restore Report for any completed restore by clicking on the restore of interest in the Activity tab.
Note: If you are restoring (essentially cloning) from an S3 backup and you close the Restore Report dialog, you must reopen the Restore Report from the destination cluster.

If there is insufficient disk space to restore the backup, the restore fails. See Monitoring sufficient disk space for restoring backups.