Using tools with TDE-encrypted SSTables

Steps to use tools with SSTables encrypted with TDE.

Introduction

This topic explains the steps to get tools working with SSTables that are encrypted with Transparent Data Encryption (TDE). You may need to set one or more values, most likely by exporting environment variables in the shell from which you run the tools like sstabledump, sstablerepairedset, and sstableloader.

Steps to get tools working with TDE-encrypted SSTables

If TDE is configured, the two options to manage encryption keys are either:
  • locally storing the keys in the cluster itself (that is, local storage)
  • or using a KMIP server
When local storage is used, the tool needs to query the cluster itself in order to retrieve the necessary encryption key or keys to work with the encrypted SSTable file or files on local disk. Because most of the DSE SSTable tools require stopping the local DSE process on the node before using a tool, the tool needs to connect to a different node in the cluster. You must provide the IP address of that different node. Do this by exporting a value for DSE_HOST environment variable. Example: 
export DSE_HOST=12.34.56.789
Tip: If KMIP is used for key management, this step is not necessary.

Setting authentication credentials

In addition to TDE, you may also configure authentication. If TDE local storage is in use, and username/password authentication is in use, supply those credentials in addition to the DSE_HOST value. DSE internal authentication and LDAP are two scenarios where username/password may be required, depending on your requirements. Using Kerberos is an example of authentication not requiring username/password credentials.

If username/password are required, provide the values via the DSE_USERNAME and DSE_PASSWORD environment variables. Example: 
export DSE_USERNAME=myusername
export DSE_PASSWORD=mysupersecretpassword
Alternatively, DSE retrieves the username and password values from the .dserc file, if present.

Additional considerations

When TDE local storage is in place, and the tool being used encounters an encrypted SSTable file, the tool uses the DSE Java driver to make a connection to a remote DSE node in the cluster, and submit a CQL query to retrieve the necessary encryption key. Again, the tool (such as sstabledump, sstablerepairedset, sstableloader) must connect to a remote node because in most cases the local DSE process needs to be stopped before any of the SSTable-related tools can be used. Thus the need for these connection credentials: the tool essentially becomes a client, via the driver, to the cluster.

DSE converts the three environment variables mentioned above by dse.in.sh shell script to Java system properties:
  • DSE_HOST becomes -Ddse.replicatedkeyprovider.client
  • DSE_USERNAME becomes -Ddse.replicatedkeyprovider.username
  • DSE_PASSWORD becomes -Ddse.replicatedkeyprovider.password
ReplicatedKeyProvider is the DSE Java class that consumes these values and handles retrieving the encryption key(s) from the cluster. DSE implements a log statement in this class that is helpful for debugging. It is logged at INFO level and available in the system.log file. Example log entry format: 
"Checking for 'dse.replicatedkeyprovider' system properties, found hostname: %s; username: %s; [no|non-null] password"