Backup Management and Restoring from Backups¶
With these methods, you can run backups immediately, view the backups that currently exist in the cluster, and restore from a backup. Scheduling a backup to run periodically can be done through the scheduling api: Schedule management
Managing Backups¶
- GET /{cluster_id}/backup-activity¶
Get a list of backups that exist in the cluster.
Path arguments: cluster_id – The ID of a cluster returned from
GET /cluster-configs.Opt. params: - count – Number of backup events to return.
- reverse – If set to 1, reverse the order results are returned.
- timestamp – Events that happened <= timestamp will be returned. The reverse flag changes this to be >=.
- filter_type – Return only events of the specified type.
- filter_live – If set to 1, return only backups that have not been deleted.
- filter_success – If set to 1, return only events that have successfully completed.
- first_backup_id – If provided any records before the indicated backup_id will not be included in the results.
Returns a list of backup events for the cluster.
Example
curl http://127.0.0.1:8888/Test_Cluster/backup-activity?count=6
Output:
[ { week: "201711", destination: { path: "", provider: "server", id: "OPSC_ON_SERVER" }, type: "backup", deleted_at: null, backup_id: "opscenter_adhoc_2017-03-17-15-06-09-UTC", keyspaces: { OpsCenter: [ "bestpractice_results", "rollup_state", "settings", "pdps", "rollups7200", "rollups86400", "backup_reports", "events_timeline", "rollups60", "events", "rollups300" ], }, event_time: 1489763169, status: "success" }, { week: "201711", destination: { throttle_bytes_per_second: "0", delete_this: "False", path: "orion-backups-2", server_side_encryption: false, provider: "s3", id: "140a4904e0da4e4e9e3dcbf8d21b56d6" }, full_status: { destinations: [ "140a4904e0da4e4e9e3dcbf8d21b56d6" ], started: 1489763169, finished: null, cluster_id: "Test_Cluster", keyspaces: { OpsCenter: [ "bestpractice_results", "rollups60", "events" ], }, details: { message: "" subrequests: { 127.0.0.1: { started: 1489763169, finished: null, cluster_id: null, details: { metadata: { created: false, errors: [ ] }, sstables: { OpsCenter: { bestpractice_results: { done_files: 0, total_files: 0, total_size: 0, errors: [ ] }, rollups60: { done_files: 0, total_files: 6, total_size: 88123, errors: [ ] }, events: { done_files: 0, total_files: 6, total_size: 5677, errors: [ ] } } }, id: "1789233b-b9b0-450d-bfaf-42729a3776e1", state: "running" } }, }, id: "777452d0-5b40-4fb7-bd66-08c02b462476", state: "running", tag: "opscenter_adhoc_2017-03-17-15-06-09-UTC" }, type: "backup", deleted_at: null, backup_id: "opscenter_adhoc_2017-03-17-15-06-09-UTC", request_id: "777452d0-5b40-4fb7-bd66-08c02b462476", event_time: 1489763169, status: "running" } ]
- GET /{cluster_id}/backups¶
Get a list of backups that exist in the cluster.
Path arguments: cluster_id – The ID of a cluster returned from
GET /cluster-configs.Opt. params: - match – A Job Schedule ID to limit backups to those created by a particular backup schedule.
- last_seen – A Unix timestamp to limit backups to those created before the timestamp. Use with amount (below) to page the results of this query.
- amount – Number of backups to return.
- destination – Id of a destination to get the snapshots from
- list_all – If set to 1, a single agent will get the information for all backups on a destination
Returns a dictionary describing backups in the cluster. Each key in the dictionary is the ID of the backup and each value is a dictionary describing the backup.
Example
curl http://127.0.0.1:8888/Test_Cluster/backups?last_seen=1334857272
Output:
{ "opscenter_adhoc_2012-04-19-17-41-12-UTC": { "id": "adhoc", "keyspaces": { "OpsCenter": { "cfs": [ "events", "events_timeline", "rollups60", ... ], "nodes": [ "127.0.0.1" ], "size": 23180 }, "system": { "cfs": [ "LocationInfo", "Schema", ... ], "nodes": [ "127.0.0.1" ], "size": 48961 }, "test": { "cfs": [ "cf1", "cf2" ], "nodes": [ "127.0.0.1" ], "size": 4543 } }, "time": 1334857272 } }
- GET /{cluster_id}/backups/{ks_name}¶
Get a list of backups that exist for a keyspace in the cluster.
Path arguments: - cluster_id – The ID of a cluster returned from
GET /cluster-configs. - ks_name – The keyspace scheduled for backup.
Returns data in the same format as
GET /{cluster_id}/backups.- cluster_id – The ID of a cluster returned from
- GET /{cluster_id}/backups/{ks_name}
Get a list of backups that exist for a keyspace in the cluster.
Path arguments: - cluster_id – The ID of a cluster returned from
GET /cluster-configs. - ks_name – The keyspace scheduled for backup.
Returns data in the same format as
GET /{cluster_id}/backups.- cluster_id – The ID of a cluster returned from
- POST /{cluster_id}/backups/run¶
Run a one time backup immediately.
Path arguments: cluster_id – The ID of a cluster returned from GET /cluster-configs.Body: A JSON dictionary containing the options to use for running this one-time backup. Returns a Request ID
Example: Store backup locally
Start a backup of two keyspaces immediately.
curl -X POST http://127.0.0.1:8888/Test_Cluster/backups/run -d '{"keyspaces": ["Keyspace1", "Keyspace2"]}'Output:
Returns a Request ID.
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"Example: Store backup to a destination
Start a backup of two keyspaces immediately to a destination with destination_id. You can backup to multiple destinations by passing in additional destination IDs to the destinations object. It’s important to note, however, that any destinations specified in the destinations object will be deleted after the backup has completed.
curl -X POST http://127.0.0.1:8888/Test_Cluster/backups/run -d '{"keyspaces": ["Keyspace1", "Keyspace2"], "destinations":{"fe85800f3f4043a88fbe76fc45b22b19":{}}}'Output:
Returns a Request ID.
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"Note: You can schedule a backup in the future using the
POST /{cluster_id}/job-schedules. Specify an interval=-1 and interval_unit=null.
- POST /{cluster_id}/backups/sync¶
- Sync a snapshot to a destination
Path arguments: cluster_id – The ID of a cluster returned from
GET /cluster-configs.Body: A JSON dictionary containing tag and destination information for the sync operation
Returns a Request ID
Example: Sync snapshot to a destination
You can backup to multiple destinations by passing in additional destination IDs to the destinations object. It’s important to note, however, that any destinations specified in the destinations object will be deleted after the backup has completed.
curl -X POST http://127.0.0.1:8888/Test_Cluster/backups/sync -d '{"tag": "opscenter_adhoc_2017-07-28-20-04-40-UTC", "destinations":{"fe85800f3f4043a88fbe76fc45b22b19":{}}}'Output:
Returns a Request ID.
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"
- DELETE /{cluster_id}/backups¶
Delete a backup from a destination.
Path arguments: - tag – The ID of a backup to restore. This must match one of the IDs
returned by
GET /{cluster_id}/backups. - destination – The destination the backup should be deleted from. If not specified the local backup will be deleted.
- remove_destination – A boolean indicating whether the specified destination should also be deleted once the backup has been deleted. If not specified this value is assumed to be false.
Example: Deleting a backup from a destination
BACKUP='opscenter_adhoc_2015-10-15-21-48-47-UTC' DESTINATION_ID='bdcff9240b614ebda64938bafb2abb09' curl -X DELETE http://127.0.0.1:8888/Test_Cluster/backups?tag=$BACKUP&destination=$DESTINATION_ID&remove_destination=true
Output:
Returns a Request ID.
"b8a8167f-5a02-425a-afbd-036ecd71846a"- tag – The ID of a backup to restore. This must match one of the IDs
returned by
- GET /{cluster_id}/backups/pit¶
Returns the current commitlog archiving config
Path arguments: cluster_id – The ID of a cluster returned from GET /cluster-configs.Example
curl http://127.0.0.1:8888/Test_Cluster/backups/pit
Output:
{ "enabled": true, "destinations": { "e2721f3b8a294eeb9684cbefe7536be4": {"compressed": false }, "OPSC_ON_SERVER": {"compressed": false } } }
- POST /{cluster_id}/backups/pit¶
Enable commitlog archiving
Path arguments: cluster_id – The ID of a cluster returned from
GET /cluster-configs.Body: A JSON dictionary of options, including:
destinations: A dictionary describing the destinations that commitlog backups should be copied to.backup_directory: The directory on the nodes where the commitlog backups should be copied to.sleep: Number of seconds to sleep between each node during the rolling restart after the change has been applied.drain_first: Controls whether nodes are drained before they are restarted.
Returns the request ID of the rolling restart triggered after enabling commitlog backups.
Example
curl -X POST http://127.0.0.1:8888/Test_Cluster/backups/pit -d '{ "destinations": { "OPSC_ON_SERVER": { "compressed": false }, "9e626c6098ab48e9bcfb8f3d60dbf9ff": { "compressed":true, "throttle":10 } }, "backup_directory": "/var/tmp", "sleep":5, "drain_first":false }'
Output:
"55851b8b-fe06-4cf3-8981-5b81f86bb06d"
- DELETE /{cluster_id}/backups/pit¶
Disable commitlog archiving
Path arguments: cluster_id – The ID of a cluster returned from
GET /cluster-configs.Body: A JSON dictionary of options, including:
sleep: Number of seconds to sleep between each node during the rolling restart after the change has been applied.drain_first: Controls whether nodes are drained before they are restarted.
Returns the request ID of the rolling restart triggered after disabling commitlog backups.
Example
curl -X DELETE http://127.0.0.1:8888/Test_Cluster/backups/pit -d '{ "sleep": 5, "drain_first": false }'
Output:
"55851b8b-fe06-4cf3-8981-5b81f86bb420"
- GET /{cluster_id}/backups/pit/logs¶
Returns a dictionary of each node and a timestamp in seconds of its most recently backed up commitlog. This information lets the user know what the most recent time is that a point in time restore can be executed to.
Path arguments: cluster_id – The ID of a cluster returned from GET /cluster-configs.Example
curl -X GET http://127.0.0.1:8888/Test_Cluster/backups/pit/logs
Output:
{ "10.240.0.3": 1450135082.502205, "10.240.0.4": 1450135124.02664 }
Restoring from Backups¶
- POST /{cluster_id}/backups/restore/{tag}¶
Restore all data from a backup.
Path arguments: - cluster_id – The ID of a cluster returned from
GET /cluster-configs. - tag – The ID of a backup to restore. This must match one of the IDs
returned by
GET /{cluster_id}/backups.
Body: A JSON dictionary of options, including:
throttle: The streaming limit in MB/second. The default behavior is no throttle.excluded-nodes: The list of nodes to exclude from the restore process.force: True or false. True ignores problems, such as some nodes or agents being down, that normally abort restoration.force_schema: True or false. True ignores schema problems that normally abort restoration.destination: (optional) Destination ID to restore from. Leave blank if restoring from on server. Must be a destination ID in the cluster config (GET /{cluster_id}/backups/destinations)keyspaces: A dictionary mapping keyspace names to a dictionary of options specific to that keyspace. If this parameter is omitted, all keyspaces will be restored. If it is present, only the keyspaces listed in the dictionary will be restored. The set of accepted options per-keyspace include the following:column-families: A list of specific tables to restore. If omitted, all tables will be restored.truncate: True or false. True first truncates all tables that will be restored. By default, tables are not truncated before restoration.
Returns a Request ID.
Example: Restore from on server
BACKUP='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC' curl -X POST http://192.168.1.1:8888/Test_Cluster/backups/restore/$BACKUP -d '{ "throttle": 10, "excluded-nodes": [ "192.168.100.7" ], "keyspaces": { "Keyspace1": { "column-families: ["users", "dates"], "truncate": true }, "OpsCenter": { "truncate": false } }, }'
Output:
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"Example: Restore from a destination
BACKUP='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC' curl -X POST http://192.168.1.1:8888/Test_Cluster/backups/restore/$BACKUP -d '{ "destination": "fe85800f3f4043a88fbe76fc45b22b19", "throttle": 10, "excluded-nodes": [ "192.168.100.7" ], "keyspaces": { "Keyspace1": { "column-families: ["users", "dates"], "truncate": true }, "OpsCenter": { "truncate": false } }, }'
Output:
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"- cluster_id – The ID of a cluster returned from
- POST /{cluster_id}/backups/restore/{tag}/{ksname}¶
Restore a keyspace from a backup.
Path arguments: - cluster_id – The ID of a cluster returned from
GET /cluster-configs. - tag – The ID of a backup to restore. This must match one of the ID’s
returned by
GET /{cluster_id}/backups. - ksname – A keyspace to restore.
Body: A JSON dictionary of options, including:
throttle: The streaming limit in MB/second. The default behavior is no throttle.excluded-nodes: The list of nodes to exclude from the restore process.force: True or false. True ignores problems, such as some nodes or agents being down, that normally abort restoration.destination: (optional) Destination ID to restore from. Leave blank if restoring from on server. Must be a destination ID in the cluster config (GET /{cluster_id}/backups/destinations)column-families: A list of specific tables to restore. If omitted, all tables will be restored.truncate: True or false. True first truncates all tables that will be restored. By default, tables are not truncated before restoration.
Returns a Request ID.
Example: Restore a keyspace from on server
BACKUP='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC' curl -X POST http://192.168.1.1:8888/Test_Cluster/backups/restore/$BACKUP/Keyspace1 -d '{ "throttle": 10, "excluded-nodes": [ "192.168.100.7" ], "column-families: ["users", "dates"], "truncate": true }'
Output:
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"Example: Restore a keyspace from a destination
BACKUP='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC' curl -X POST http://192.168.1.1:8888/Test_Cluster/backups/restore/$BACKUP/Keyspace1 -d '{ "throttle": 10, "destination": "fe85800f3f4043a88fbe76fc45b22b19", "excluded-nodes": [ "192.168.100.7" ], "column-families: ["users", "dates"], "truncate": true }'
Output:
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"- cluster_id – The ID of a cluster returned from
- POST /{cluster_id}/backups/restore/{tag}/{ksname}/{cfname}¶
Restore a table from a backup.
Path arguments: - cluster_id – The ID of a cluster returned from
GET /cluster-configs. - tag – The ID of a backup to restore. This must match one of the ID’s
returned by
GET /{cluster_id}/backups. - ksname – A keyspace to restore.
- cfname – A table name to restore.
Body: A JSON dictionary of options, including:
throttle: The streaming limit in MB/second. The default behavior is no throttle.excluded-nodes: The list of nodes to exclude from the restore process.force: True or false. True ignores problems, such as some nodes or agents being down, that normally abort restoration.destination: (optional) Destination ID to restore from. Leave blank if restoring from on server. Must be a destination ID in the cluster config (GET /{cluster_id}/backups/destinations)truncate: True or false. True first truncates all tables that will be restored. By default, tables are not truncated before restoration.
Returns a Request ID.
Example: Restore a specific table from a Keyspace on server
BACKUP='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC' curl -X POST http://192.168.1.1:8888/Test_Cluster/backups/restore/$BACKUP/Keyspace1/users -d '{ "throttle": 10, "excluded-nodes": [ "192.168.100.7" ], "truncate": true }'
Output:
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"Example: Restore a specific table from a keyspace from a destination
BACKUP='opscenter_4a269167-96c1-40c7-84b7-b070c6bcd0cd_2012-06-07-18-00-00-UTC' curl -X POST http://192.168.1.1:8888/Test_Cluster/backups/restore/$BACKUP/Keyspace1/users -d '{ "throttle": 10, "destination": "fe85800f3f4043a88fbe76fc45b22b19", "excluded-nodes": [ "192.168.100.7" ], "truncate": true }'
Output:
"fee2232c-48ac-11e2-b1b9-e0b9a54a6d93"- cluster_id – The ID of a cluster returned from
- POST /{cluster_id}/backups/pit_check¶
Validate a point in time to be restored to
Path arguments: cluster_id – The ID of a cluster returned from
GET /cluster-configs.Body: A JSON dictionary of options, including:
pit: The timestamp in seconds to restore to.destination: Destination containing the snapshot to use for the restore.
Example
curl -X POST http://127.0.0.1:8888/Test_Cluster/backups/pit_check -d '{ "pit": 1450143120, "destination": "OPSC_ON_SERVER" }'Output:
{ "message": "No snapshots found before date and time: 12-15-2015 01:32:00 UTC", "type": "InvalidArguments", "brief": "error" }
- POST /{cluster_id}/backups/pit/run/¶
Run a point in time restore
Path arguments: cluster_id – The ID of a cluster returned from
GET /cluster-configs.Body: A JSON dictionary of options, including:
keyspaces: Dictionary of keyspaces to restore from the snapshot.destination: (optional) Destination ID to restore from. Leave blank if restoring from on server. Must be a destination ID in the cluster config (GET /{cluster_id}/backups/destinations).pit_destination: Destination containing the commitlogs to use for the restore.pit: The timestamp in seconds to restore to.restart_pause: (optional) The number of seconds to wait between restarting each node
Example
curl -X POST http://127.0.0.1:8888/Test_Cluster/backups/pit_check -d '{ "keyspaces": { "Keyspace1": { "column-families: ["users", "dates"], "truncate": true }, "OpsCenter": { "truncate": false } }, "pit_destination": "OPSC_ON_SERVER", "pit":1450144260 }'
Output:
{ "message": "No snapshots found before date and time: 12-15-2015 01:32:00 UTC", "type": "InvalidArguments", "brief": "error" }
- GET /{cluster_id}/backups/graphs/{tag}/{destination_id}¶
Get a map of graphs to their associated keyspaces as they exist in the specified snapshot.
Path arguments: - cluster_id – The ID of a cluster returned from
GET /cluster-configs. - tag – The tag of the backup snapshot as from the key values returned by (
GET /{cluster_id}/backups) - destination_id – Optional ID of the backup destination. Must be a destination ID in the cluster config (
GET /{cluster_id}/backups/destinations). Default is the On Server destination.
Returns a dictionary of the keyspaces owned by a particular graph.
Example
curl http://127.0.0.1:8888/Test_Cluster/backups/graphs/opscenter_adhoc_2017-03-17-15-06-09-UTC/06747042-5a9a-45e7-8fa0-297abb68448f
Output:
{ "graph1": ["graph1_ks1", "graph1_ks2"] "graph2": ["graph2_ks1", "graph2_ks2", "graph2_ks3"] }
- cluster_id – The ID of a cluster returned from
Managing Destinations¶
- GET /{cluster_id}/backups/destinations¶
Get a list of backup destinations that exist in for that cluster.
Path arguments: cluster_id – The ID of a cluster returned from GET /cluster-configs.Returns a dictionary describing the destinations in the cluster. Each key in the dictionary is the ID of the backup destination.
Example
curl http://127.0.0.1:8888/Test_Cluster/backups/destinations
Output:
{ "06747042-5a9a-45e7-8fa0-297abb68448f": { "access_key": "AKIAIDGKF45VQXGUYIQQ", "access_secret": "gMizFwIoFHzIo7RrNmXUUd9zKea8hSkh6+wXLCT1", "path": "example-opscenter-backups-1", "provider": "s3" }, "16747042-5a9a-45e7-8fa0-297abb68448g": { "access_key": "AKIAIDGKF45VQXGUYIQQ", "access_secret": "gMizFwIoFHzIo7RrNmXUUd9zKea8hSkh6+wXLCT1", "path": "example-opscenter-backups-2", "provider": "s3" } }
- GET /{cluster_id}/backups/destinations/{destination_id}¶
Get information about a destination
Path arguments: - cluster_id – The ID of a cluster returned from
GET /cluster-configs. - destination_id – The ID of the backup destination. Must be a destination ID in the cluster config (
GET /{cluster_id}/backups/destinations)
Returns a dictionary describing the attributes of the specific backups destination in the cluster.
Example
curl http://127.0.0.1:8888/Test_Cluster/backups/destinations/06747042-5a9a-45e7-8fa0-297abb68448f
Output:
{ "access_key": "AKIAIDGKF45VQXGUYIQQ", "access_secret": "gMizFwIoFHzIo7RrNmXUUd9zKea8hSkh6+wXLCT1", "path": "example-opscenter-backups-1", "provider": "s3" }
- cluster_id – The ID of a cluster returned from
- POST /{cluster_id}/backups/destinations¶
Adds or updates a destination in the cluster. This will also perform validation on the destination before adding a destination. To update an existing destination, include “id”: “the-id”, where “the-id” is a destination listed from
GET /{cluster_id}/backups/destinations, in the JSON body. You can also pass in the key “delete_this” with a value of true to delete a destination.Path arguments: cluster_id – The ID of a cluster returned from GET /cluster-configs.Returns an array of dictionaries describing the attributes of the updated backup destinations in the cluster and the Request request_id of the request that updates all the agents with the new destination
Example: Add New Destination
curl -X POST http://127.0.0.1:8888/Test_Cluster/backups/destinations -d '{ "access_key": "AKIAIDGKF45VQXGUYIQQ", "access_secret": "gMizFwIoFHzIo7RrNmXUUd9zKea8hSkh6+wXLCT1", "path": "example-opscenter-backups-1", "provider": "s3" }'Output:
{ destination_id: { "access_key": "AKIAIDGKF45VQXGUYIQQ", "access_secret": "gMizFwIoFHzIo7RrNmXUUd9zKea8hSkh6+wXLCT1", "path": "example-opscenter-backups-1", "provider": "s3" }, request_id: "06747042-5a9a-45e7-8fa0-297abb68448f" }
Example: Edit Existing Destination
curl -X POST http://127.0.0.1:8888/Test_Cluster/backups/destinations -d '{ "id": "06747042-5a9a-45e7-8fa0-297abb68448f", "access_key": "AKIAIDGKF45VQXGUYIQQ", "access_secret": "gMizFwIoFHzIo7RrNmXUUd9zKea8hSkh6+wXLCT1", "path": "example-opscenter-backups-1", "provider": "s3", "delete_this": false }'Output:
{ destination_id: { "access_key": "AKIAIDGKF45VQXGUYIQQ", "access_secret": "gMizFwIoFHzIo7RrNmXUUd9zKea8hSkh6+wXLCT1", "path": "example-opscenter-backups-1", "provider": "s3" }, request_id: "06747042-5a9a-45e7-8fa0-297abb68448f" }
- DELETE /{cluster_id}/backups/destinations/{destination_id}¶
Deletes a backup destination for a cluster.
Path arguments: - cluster_id – The ID of a cluster returned from
GET /cluster-configs. - destination_id – The ID of the destination to delete. Must be a destination ID in the cluster config (
GET /{cluster_id}/backups/destinations)
Returns a Request ID that removes the destination from all agents
Example
curl -X DELETE http://127.0.0.1:8888/Test_Cluster/backups/destinations/fee2232c-48ac-11e2-b1b9-e0b9a54a6d93Output:
"06747042-5a9a-45e7-8fa0-297abb68448f"- cluster_id – The ID of a cluster returned from