Rekeying Existing Data

Create a new local encryption key, change the table key filename, and re-encrypt the SSTables using the new key. When changing the system key, all existing data must be re-encrypted before removing the old key.

Prerequisites

These steps require the following privileges:

DataStax Enterprise node administrator or superuser account with read/write/modify permission on DSE resources and configuration directories.
When DSE database authentication and authorization is enabled, a database account with ALTER TABLE permission on the encrypted tables.

Procedure

Back up SSTables.

Create a new local encryption key and distribute to nodes in the cluster:

Go to the key file directory, which is defined in the system_key_directory setting of the dse.yaml.

The location of the dse.yaml file depends on the type of installation:

Package installations: /etc/dse/dse.yaml
Tarball installations: <installation_location>/resources/dse/conf/dse.yaml

Use dsetool createsystemkey to create a new system key in the key file directory:
```
dsetool createsystemkey 'AES/ECB/PKCS5Padding' 128 <new_system_key>
```
Both the new and old key are required for re-encryption. Do NOT remove the old key until after changing the table schema and rekeying the existing SSTables.
Verify that the database account has read and write access to the files:
```
la -l
```
In this example DSE is the account that runs the database.
```
-rw------- 1 dse dse 50 May 19 10:54 system_key
-rw------- 1 dse dse 50 May 19 11:20 new_system_key
```
Copy the new key to the system key directory on all nodes in the cluster.

Ensure that the new key has the correct permissions.

Change the key filename in the table schemas:
1. Get a list of all the encrypted tables that you want to change.
  
  Use the DESC KEYSPACE keyspace_name cqlsh command to show all table properties in a keyspace.
2. For each table where you want use the new encryption key, change the key filename in the table schema:
  ALTER TABLE <keyspace_name>.<table_name> WITH compression = { 'sstable_compression' : 'EncryptingSnappyCompressor', 'cipher_algorithm' : 'AES/ECB/PKCS5Padding', 'secret_key_strength' : 128, 'chunk_length_kb' : 128, 'system_key_file':'<new_system_key>' }
3. Ensure that the schema change has replicated to all nodes in the cluster:
  nodetool describecluster
  The following example shows a small three node cluster where all three nodes have the same schema. If any of the nodes have a different schema, then wait until the schema changes are propagated before continuing with the next step.
  Name: Cluster1 Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Schema versions: 25743512-6b6f-3f76-96bc-1122d441f539: [<node1_IP>, <node2_IP>, <node3_IP>]
Use nodetool upgradesstables to rewrite the encrypted SSTables using the new key. Run the following command on every node in the cluster:
- Target only specific tables:
  nodetool upgradesstables --include-all-sstables keyspace_name table_name [table_name …]
- Target specific keyspace:
  nodetool upgradesstables --include-all-sstables keyspace_name
- All keyspaces and tables:
  nodetool upgradesstables --include-all-sstables
After completing the prior steps, remove the old key and ensure that the old key is not used for any tables or configuration file property encryption.

The old key is required to access the backed up SSTables created in the first step.