Configuring KMIP encryption
Set up encryption using keys from a KMIP (Key Management Interoperability Protocol) host to protect sensitive configuration file properties, system resources, and tables.
DataStax recommends using KMIP key server security policies to limit the number of nodes in the cluster that can remotely manage keys, due to the risks associated with expiring, revoking, and destroying keys. |
Use OpsCenter to monitor KMIP server status. See Configuring an alert for KMIP errors.
Adding a KMIP host
DataStax Enterprise supports using encryption keys from one or more remote KMIP hosts to encrypt/decrypt table data and/or sensitive properties in the dse.yaml and cassandra.yaml configuration files. Follow these steps to add a KMIP server information to the list of available hosts.
DataStax recommends limiting the number of nodes that can remotely manage KMIP keys using a security policy on the KMIP host.
Procedure
Perform all steps on every node in the cluster.
-
Set up KMIP agents and registered DSE with the KMIP service:
Refer to the KMIP key provider documentation for detailed steps.
-
Download and install the KMIP agent.
-
Connect to the KMIP host.
-
Register the DSE node.
-
Locate the SSL key pair generated by the KMIP agent.
-
-
Convert the key pair from PEM to a DSE compatible JKS format:
-
Secure the KMIP agent private key files by removing read access for all users, for example the Vormetric DSM agents creates two files named
<kmip-key>.pem
and<kmip-host_name>.pem
. -
Copy both keys to another directory, such as your home directory.
-
Generate a PKCS12 format file from the PEM files:
openssl pkcs12 -export -out <kmip_keystore>.p12 -inkey <kmip-key>.pem -in <kmip-host_name>.pem
Where <kmip_keystore>.p12 is the output file name and <kmip-host_name>.pem is part of the key pair created by the KMIP agent.
-
Create a JKS keystore:
keytool -importkeystore -destkeystore <kmip_keystore.jks> -srcstoretype PKCS12 -srckeystore <kmip_keystore.p12>
Where
-
<kmip_keystore.jks> is the keystore file name that is created
-
<kmip_keystore.p12> is the PKC12 file generated in the previous step
Enter a password for the keystore at the prompt and fill out the host information.
-
-
Install the KMIP root certificate into the JKS truststore:
keytool -import -alias <kmipCA> -file <kmip-host_CA.pem> -keystore <kmip_truststore.jks>
Enter a password for the truststore at the prompt and fill out the host information.
-
Move the keystore and truststore to a directory accessible by DSE and change the file to allow the DSE account read/write access.
-
Delete or secure the files used to create the keystore and truststore.
-
-
Add the host details to the
kmip_hosts
section of the dse.yaml:kmip_hosts: <kmip_group_name>: hosts: <FQDN>[, <FQDN> , ...] keystore_path: </etc/dse/conf/kmip_keystore.jks> keystore_type: jks keystore_password: <password> truststore_path: </etc/dse/conf/kmip_truststore.jks> truststore_type: jks truststore_password: <password> key_cache_millis: <N> timeout: <N> protocol: <protocol> cipher_suites: <supported_cipher>
-
Required settings:
-
<kmip_group_name>: User-defined group name that identifies the KMIP host in DSE related commands.
-
hosts
: Comma separated list of fully qualified domain names (FQDN) of KMIP hosts. DSE tries the hosts in the order listed. -
keystore_path
: Location of the keystore created in step 2. -
keystore_type: jks
Keystore format. Must be set tojks
. -
keystore_password
: Password of the keystore file created in step 2. -
truststore_path
: Location of the truststore file created in step 2. -
truststore_type: jks
Truststore format. Must be set tojks
. -
truststore_password
: Password of the truststore file created in step 2.
-
-
Optional settings:
-
key_cache_millis: <N>
where N is the interval at which DSE refreshes the key cache on the node in milliseconds. The default is 300000 (five minutes). -
timeout: <N>
where N is the socket timeout in milliseconds. The default is 1000. -
protocol: <protocol>
for communicating between the node and KMIP key server. When not specified, JVM default is used. -
cipher_suites: https:/docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/config/configDseYaml.html#configDseYaml__cipher_suites[supported_cipher]
for communicating between the node and KMIP key server. When not specified, JVM default is used.
-
-
-
Verify that the node can connect to the KMIP host by listing encryption keys on the remote KMIP server:
dsetool managekmip list <kmip_group_name>
dsetool
picks up dse.yaml changes without requiring a restart.If problems connecting to the KMIP server occur, see Troubleshooting KMIP connections.
-
Repeat steps on all nodes in the cluster.
Encrypting configuration file properties
Configure DSE to use a KMIP encryption key to decrypt sensitive configuration properties. Use passwords encrypted with the KMIP key for the following properties:
-
dse.yaml LDAP values:
ldap_options.search_password ldap_options.truststore_password
Restriction: Use plain text for the KMIP keystore or truststore passwords.
-
cassandra.yaml SSL values:
server_encryption_options.keystore_password server_encryption_options.truststore_password client_encryption_options.keystore_password client_encryption_options.truststore_password
DataStax Enterprise caches encryption keys from the KMIP host and refreshes the cache at the interval set by the |
Prerequisites
Complete the steps in Adding a KMIP host.
If any of the defined KMIP groups are not available, DSE startup fails. |
Prerequisites
Complete the steps in Adding a KMIP host.
If any of the defined KMIP groups are not available, DSE startup fails. |
Procedure
-
Back up the configuration files.
-
Get the URL of the KMIP encryption key:
-
To create a new key and get the URL:
dsetool createsystemkey '<AES/ECB/PKCS5>' 128 -k <kmip_group_name>
The example output shows the URL for the
<host_name>
in the dse.yaml with ID: 02-1655.kmip://<host_name>/02-1655
-
To use an existing KMIP key, the URL syntax is
kmip://<kmip_group_name>/<ID>
. To look up the key ID:dsetool managekmip list <kmip_group_name>
For example, the
<host_name>
has the following keys:Keys on <host_name>: ID Name Cipher State Activation Date Creation Date Protect Stop Date Namespace 02-449 82413ef3-4fa6-4d4d-9dc8-71370d731fe4_0 AES/CBC/PKCS5 Deactivated Mon Apr 25 20:25:47 UTC 2016 n/a n/a n/a 02-540 0eb2277e-0acc-4adb-9241-1dd84dde691c_0 AES Active Tue May 31 12:57:59 UTC 2016 n/a n/a`
The URL of the second key in the list is
kmip://<host_name>/02-540
.
-
-
Configure system property encryption settings in the dse.yaml.
-
Enable system property encryption:
config_encryption_active: true
-
Set the URL of the KMIP key used to decrypt properties:
config_encryption_key_name: <KMIP_key_URL>
Where <KMIP_key_URL> format is
kmip://kmip_group_name/key-id
, for examplekmip://<host_name>/02-1655
.
-
-
For each property, replace plain text passwords with encrypted passwords returned by running the dsetool encryptconfigvalue command:
-
Encrypt the password:
dsetool encryptconfigvalue
Using system key system_key Enter value to encrypt: Enter again to confirm: Your encrypted value is: +Vj5oHCR/jqfA+OJE2m8zA==
-
Replace the old value with the new value in the configuration file, for example the SSL truststore password in the cassandra.yaml:
truststore_password: +Vj5oHCR/jqfA+OJE2m8zA==
Once configuration file property encryption is enabled, DSE startup fails if any of the protected properties are not encrypted.
-
-
Optional. Set up system resource encryption.
Encrypting system resources
Use a KMIP key to encrypt the system.batches
and system.paxos
tables, hint files and commit logs.
Prerequisites
Complete the steps in Adding a KMIP host.
If any of the defined KMIP groups are not available, DSE startup fails. |
Procedure
-
In the dse.yaml file, configure encryption settings for system tables, the commit log, and the hints files.
system_info_encryption: enabled: ( true | false ) cipher_algorithm: <cipher_name> secret_key_strength: <length> key_provider: KmipKeyProviderFactory kmip_host: <kmip_group_name> chunk_length_kb: 64
-
Required properties:
-
enabled
: Set totrue
. On the next startup, system resources are encrypted. If the system tables have existing data, use nodetool upgradesstables to apply encryption. -
key_provider
: Set toKmipKeyProviderFactory
. -
kmip_host
: Use the group name from the kmip_hosts section.
-
-
Optional. To ensure that KMIP generates a compatible key, configure the type of encryption key to use:
-
cipher_algorithm
: Set the name of a supported JCE cipher algorithm to use. DSE supports the following algorithms:cipher_algorithm secret_key_strength AES
128, 192, or 256
DES
56
DESede
112 or 168
Blowfish
32-448
RC2
40-128
-
secret_key_strength
: Specify the key length. -
chunk_length_kb
: Configures chuck size for SSTables. The default (64) is used if the option is excluded. When these properties are set, DSE only uses a key that matches; if no matching key exists, start up fails.
-
-
-
To encrypt existing data, run
nodetool upgradesstables -a system batchlog paxos
on all nodes in the cluster.
Encrypting table data
Encrypt data stored in a table data using a KMIP key.
Primary keys are stored in plain text. Do not put sensitive information in partition key or clustering columns. |
Prerequisites
Complete the steps in Adding a KMIP host.
If any of the defined KMIP groups are not available, DSE startup fails. |
Procedure
-
To create a new encrypted table using a key from a KMIP server:
-
Encryption without compression:
CREATE TABLE customers ... WITH COMPRESSION = { 'class': 'Encryptor', 'key_provider': 'KmipKeyProviderFactory', 'kmip_host': '<kmip_group_name>' ['key_namespace' = '<kmip_namespace>'], 'cipher_algorithm': 'AES/ECB/PKCS5Padding', 'secret_key_strength': 128 };
-
'key_provider': 'KmipKeyProviderFactory'
tells the encryptor to use a KMIP key server to manage its encryption keys. Include the'key provider'
entry only to specify to use a KMIP key server, otherwise omit this entry. -
'kmip_host': '<kmip_group_name>'
specifies the user-defined KMIP key server group name defined in the 'kmip_hosts' section of dse.yaml. -
'kmip_host': '<kmip_group_name>' ['key_namespace' = '<kmip_namespace>'] specify an optional KMIP namespace. Using namespaces allows you to granularly manage keys on a per table or keyspace basis.
-
-
Compression and encryption:
CREATE TABLE customers ... WITH COMPRESSION = { 'class': 'EncryptingDeflateCompressor', 'key_provider': 'KmipKeyProviderFactory', 'kmip_host': '<kmip_group_name>', 'cipher_algorithm': 'AES/ECB/PKCS5Padding', 'secret_key_strength': 128 };
-
-
To encrypt an already existing table:
-
Change the table compression settings:
-
Encryption without compression:
ALTER TABLE customers ... WITH COMPRESSION = { 'class': 'Encryptor', 'key_provider': 'KmipKeyProviderFactory', 'kmip_host': '<kmip_group_name>' ['key_namespace' = '<kmip_namespace>'], 'cipher_algorithm': 'AES/ECB/PKCS5Padding', 'secret_key_strength': 128 };
-
'key_provider': 'KmipKeyProviderFactory'
tells the encryptor to use a KMIP key server to manage its encryption keys. Include the'key provider'
entry only to specify to use a KMIP key server, otherwise omit this entry. -
'kmip_host': '<kmip_group_name>'
specifies the user-defined KMIP key server group name defined in thekmip_hosts
section of dse.yaml. -
['key_namespace' = '<kmip_namespace>'] specify an optional KMIP namespace. Using namespaces allows you to granularly manage keys on a per table or keyspace basis.
-
-
Compression and encryption:
ALTER TABLE customers ... WITH COMPRESSION = { 'class': 'EncryptingDeflateCompressor', 'key_provider': 'KmipKeyProviderFactory', 'kmip_host': '<kmip_group_name>', 'cipher_algorithm': 'AES/ECB/PKCS5Padding', 'secret_key_strength': 128 };
-
-
Encrypt existing data on all nodes in the cluster:
nodetool upgradesstables -a [<keyspace_name> [<table_name>[ <tablename>]...]
-
Expiring an encryption key
Security policies generally limit the amount of time an encryption key is in use; this section describes how to expire a key without re-encrypting the exiting data. After a key expires, it is no longer used to encrypt new data, but is still used to decrypt existing data.
To change the key used for both encryption and decryption, see Rekeying tables using a new key. |
Procedure
-
Get a list of the available keys and states from the KMIP server:
dsetool managekmip list <kmip_group_name>
For example, the <host_name> has two keys:
Keys on <host_name>: ID Name Cipher State Activation Date Creation Date Protect Stop Date Namespace 02-449 82413ef3-4fa6-4d4d-9dc8-71370d731fe4_0 AES/CBC/PKCS5 Deactivated Mon Apr 25 20:25:47 UTC 2016 n/a n/a n/a 02-540 0eb2277e-0acc-4adb-9241-1dd84dde691c_0 AES Active Tue May 31 12:57:59 UTC 2016 n/a n/a
DSE supports one or more KMIP hosts. Each KMIP host is defined under a user-defined kmip_group_name in the
kmip_hosts
section of the dse.yaml. -
Expire the key:
-
Immediately expire the key:
dsetool managekmip expirekey <kmip_group_name> <key_id>
-
Schedule an expiration date:
dsetool managekmip expirekey <kmip_group_name> <key_id> <datetime>
After the key expires, the database gets a new key for encryption the next time it refreshes the key cache (
key_cache_millis
in dse.yaml); the default setting is five minutes. Expired keys are still available to decrypt data.After the key expires, the database gets a new key for encryption the next time it refreshes the key cache (
key_cache_millis
in dse.yaml); the default setting is five minutes. Expired keys are still available to decrypt data.
-
-
Optionally, force a refresh of the DSE key cache by performing a rolling restart.
Rekeying tables using a new key
Change the encryption key that is used for both encrypting new data and decrypting the existing data. Use these steps to secure the data after an event that potentially compromised an encryption key, such as a change in security administration staff. Before destroying the old key, revoke the compromised KMIP key, wait for the database key cache refresh, and then re-encrypt existing SSTables with the new key.
The database caches the encryption keys and refreshes the cache at an interval set by the key_cache_millis
in dse.yaml (default setting is 5 minutes).
To get a new key, either wait for the key cache refresh interval or perform a rolling restart.
The dsetool managekmip commands proxy KMIP commands to the corresponding host; the commands effect encryption keys as follows:
|
Procedure
-
Revoke the compromised key using the dsetool:
-
Get the ID of the KMIP encryption key you want to revoke from the KMIP server:
dsetool managekmip list <kmip_groupname>
The following is an example of a KMIP server that has two keys, active and deactivated.
ID Name Cipher State Activation Date Creation Date Protect Stop Date Namespace 02-449 82413ef3-4fa6-4d4d-9dc8-71370d731fe4_0 AES/CBC/PKCS5 Deactivated Mon Apr 25 20:25:47 UTC 2016 n/a n/a n/a 02-540 0eb2277e-0acc-4adb-9241-1dd84dde691c_0 AES Active Tue May 31 12:57:59 UTC 2016 n/a n/a n/a
DSE supports one or more KMIP hosts. Each KMIP host is defined under a user-defined kmip_group_name in the
kmip_hosts
section of the dse.yaml. -
Revoke the key you want to replace using the ID:
dsetool managekmip expirekey <kmip_groupname> <key_id>
Revoking permanently deactivates the key on the KMIP server. When the key cache refreshes a new key for encryption/decryption is automatically created. Revoked keys are used to decrypt existing data. DO NOT destroy the revoked key until after re-encrypting the existing data.
-
Verify that the key State is Deactivated.
dsetool managekmip list <kmip_groupname>
The following is an example of a KMIP server that has two keys:
ID Name Cipher State Activation Date Creation Date Protect Stop Date Namespace 02-449 82413ef3-4fa6-4d4d-9dc8-71370d731fe4_0 AES/CBC/PKCS5 Deactivated Mon Apr 25 20:25:47 UTC 2016 n/a n/a n/a 02-540 0eb2277e-0acc-4adb-9241-1dd84dde691c_0 AES Deactivated Tue May 31 12:57:59 UTC 2016 n/a Thu Jul 27 17:16:38 UTC 2017
-
-
Refresh the database key cache using one of the following methods:
-
Wait the amount of time specified in the
key_cache_millis
in dse.yaml setting before continuing to the next step. -
Perform a rolling restart, see Starting and stopping.
KMIP keys are cached on the DSE node. DSE refreshes the cache and a new key is automatically generated by the KMIP server after
key_cache_millis
in dse.yaml lapses; the default setting is 5 minutes.
-
-
(Optional) Get a list of the affected tables to re-encrypt using the new key using the DESC keyspace command:
For example to find all tables in the cycling keyspace that use the KMIP group:
DESC KEYSPACE cycling
-
Use nodetool upgradesstables to rewrite the encrypted SSTables using the new key. Run the following command on every node in the cluster:
-
Target only specific tables:
nodetool upgradesstables --include-all-sstables <keyspace_name> <table_name> [<table_name> …]
-
Target specific keyspace:
nodetool upgradesstables --include-all-sstables <keyspace_name>
-
All keyspaces and tables:
nodetool upgradesstables --include-all-sstables
-
-
(Optional) Remove the encryption key so that it is no longer available for decryption:
dsetool managekmip destroy <key_id>
The backed up SSTables are only accessible using the old key. Ensure that the data is accessible before removing the key.
Troubleshooting KMIP connections
The following error message may occur if the DataStax Enterprise SSL certificate is self-signed or from a CA that has not been added to the certificate chain on the KMIP server.
WARN 14:46:42,928 Unable to connect to KMIP host: 10.120.15.100:9005 com.cryptsoft.kmip.TTLVReadException: Error reading TTLV ResponseMessage. Got end-of-stream after reading 0 byte(s). Read bytes: (0) []
Ensure that trust between DSE and KMIP host has been properly configured, see Setting up SSL certificates.