Migrate encrypted tables from earlier versions

After upgrading from an earlier version of DataStax Enterprise (DSE), you must migrate encrypted tables.

This process requires a backup of all system keys and the entire dse_system keyspace, which includes the encrypted_keys table.

If the encryption keys are lost, the data is unrecoverable.

Backing up your installation is a pre-upgrade step for all DSE upgrade guides. If you didn’t create a backup of your installation before upgrading, and you have no earlier backups that you can recover from, contact IBM Support.

Check that the `encrypted_keys` table exists

After upgrading your cluster, perform a rolling restart of all nodes if you haven’t done so already.
Use DESCRIBE KEYSPACE to check whether the dse_system.encrypted_keys table was created in your new version.

If the table doesn’t exist, see Restore encrypted tables and indexes.

If the table exists, see Rewrite SSTables and rebuild indexes.

Restore encrypted tables and indexes

If the dse_system.encrypted_keys table doesn’t exist, create the schema manually with the following commands.

EverywhereStrategy is the default replication strategy for the dse_system and solr_admin keyspaces. Don’t use other strategies for these keyspaces. Don’t alter any other instances of EverywhereStrategy.

If the dse_system keyspace doesn’t exist, create it:

CREATE KEYSPACE dse_system WITH replication = {'class': 'EverywhereStrategy'};

Select the dse_system keyspace:
```
USE dse_system;
```

Create the encrypted_keys table:

CREATE TABLE encrypted_keys (
  key_file text,
  cipher text,
  strength int,
  key_id timeuuid,
  key text,
  PRIMARY KEY (key_file, cipher, strength, key_id)
);

If needed, restore the dse_system keyspace from your pre-upgrade backup.

Don’t truncate or delete anything when restoring the dse_system keyspace.
Complete the steps in Rewrite SSTables and rebuild indexes.

Rewrite SSTables and rebuild indexes

Rewrite all SSTables.
```
nodetool upgradesstables -a
```

Verify that search index files are encrypted:

dsetool list_index_files KEYSPACE_NAME.TABLE_NAME

Rebuild encrypted indexes:
Rebuild on the local node
This is a long-running operation that is best scheduled during a maintenance window or slow period, but it doesn’t require access to a remote node. To fully reindex a local node with it’s locally stored encryption configuration, use dsetool reload_core with deleteAll=true and reindex=true:
dsetool reload_core keyspace_name.table_name distributed=false deleteAll=true reindex=true
Rebuild from a remote node
Offline index encryption is fastest. However, the local node must be offline, and you must have access to a remote node that is running and has the index configuration available. The encryption configuration is read from the remote node. For options, see the dsetool upgrade_index_files reference.
dsetool upgrade_index_files KEYSPACE_NAME.TABLE_NAME -h REMOTE_NODE_IP_ADDRESS
Eventual encryption

Don’t rely on eventual encryption when migrating encrypted indexes.

Migrate encrypted tables from earlier versions

Check that the `encrypted_keys` table exists

Restore encrypted tables and indexes

Rewrite SSTables and rebuild indexes

Was this helpful?

Give Feedback

Migrate encrypted tables from earlier versions

Check that the encrypted_keys table exists

Restore encrypted tables and indexes

Rewrite SSTables and rebuild indexes

Was this helpful?

Check that the `encrypted_keys` table exists