Configuring KMIP Encryption

Set up encryption using keys from a Key Management Interoperability Protocol (KMIP) host to protect sensitive configuration file properties, system resources, and tables.

DataStax recommends using KMIP key server security policies to limit the number of nodes in the cluster that can remotely manage keys, due to the risks associated with expiring, revoking, and destroying keys.

Use OpsCenter to monitor KMIP server status. See Configuring an alert for KMIP errors.

Adding a KMIP Host

DataStax Enterprise (DSE) supports using encryption keys from one or more remote Key Management Interoperability Protocol (KMIP) hosts to encrypt or decrypt table data or sensitive properties, or both, in the dse.yaml and cassandra.yaml configuration files.

The location of each configuration file depends on the type of installation:

Configuration File Location
Filename Package installations Tarball installations

cassandra.yaml

/etc/dse/cassandra/cassandra.yaml

<installation_location>/resources/cassandra/conf/cassandra.yaml

dse.yaml

/etc/dse/dse.yaml

<installation_location>/resources/dse/conf/dse.yaml

DataStax recommends limiting the number of nodes that can remotely manage KMIP keys using a security policy on the KMIP host.

Follow these steps to add a KMIP server information to the list of available hosts. Perform all steps on every node in the cluster.

  1. Set up KMIP agents and registered DSE with the KMIP service: RSee your KMIP key provider documentation for more information.

    1. Download and install the KMIP agent.

    2. Connect to the KMIP host.

    3. Register the DSE node.

    4. Locate the SSL key pair generated by the KMIP agent.

  2. Convert the key pair from PEM to a DSE compatible JKS format:

    1. Secure the KMIP agent private key files by removing read access for all users. For example, the Vormetric DSM agents creates two files named kmip-key.pem and kmip-<host_name>.pem.

    2. Copy both keys to another directory, such as your home directory.

    3. Generate a PKCS12 format file from the PEM files:

      openssl pkcs12 -export -out <kmip_keystore>.p12 -inkey <kmip-key.pem> -in <kmip-host_name.pem>

      <kmip_keystore>.p12 is the output file name, and <kmip-host_name.pem> is part of the key pair created by the KMIP agent.

      Due to an OpenSSL issue, PKCS12 truststores generated with OpenSSL can be incompatible with DSE.

      For example, a truststore generated with the following command might not work with DSE:

      openssl pkcs12 -export -nokeys -out truststore.pfx -in <intermediate.chain.pem>

      However, if you generate a truststore with Java’s keytool, and then convert it to PKCS12, it will work with DSE. For example:

      1. Create the truststore with keytool:

        keytool -importcert -alias <rootca> -file <rootca.pem> -keystore <truststore.jks>

      2. Import the intermediate certificate:

        keytool -importcert -alias <intermediate> -file <intermediate.pem> -keystore <truststore.jks>

      3. Convert the JKS truststore to PKCS12:

        keytool -importkeystore -srckeystore <truststore.jks> -destkeystore <truststore.pfx> -deststoretype pkcs12

    4. Create a JKS keystore:

      keytool -importkeystore -destkeystore <kmip_keystore.jks> -srcstoretype PKCS12 -srckeystore <kmip_keystore.p12>

      <kmip_keystore.jks> is the keystore file name that is created, and <kmip_keystore.p12> is the PKC12 file generated in the previous step.

    5. Enter a password for the keystore at the prompt and fill out the host information.

    6. Install the KMIP root certificate into the JKS truststore:

      keytool -import -alias <kmipCA> -file <kmip-host_CA.pem> -keystore <kmip_truststore.jks>

    7. Enter a password for the truststore at the prompt and fill out the host information.

    8. Move the keystore and truststore to a directory accessible by DSE and change the file to allow the DSE account read/write access.

    9. Delete or secure the files used to create the keystore and truststore.

  3. Add the host details to the kmip_hosts section of the dse.yaml:

    kmip_hosts:
  <kmip_group_name>:
    hosts: <FQDN>[, <FQDN> , ...]
    keystore_path: </etc/dse/conf/kmip_keystore.jks>
    keystore_type: jks
    keystore_password: <password>
    truststore_path: </etc/dse/conf/kmip_truststore.jks>
    truststore_type: jks
    truststore_password: <password>
    key_cache_millis: <N>
    timeout: <N>
    protocol: <protocol>
    cipher_suites: <supported_cipher>

    • <kmip_group_name> (required): User-defined group name that identifies the KMIP host in DSE related commands.

    • hosts (required): Comma separated list of Fully-Qualified Domain Names (FQDN) of KMIP hosts. DSE tries the hosts in the order listed.

    • keystore_path (required): Location of the keystore created in 2.

    • keystore_type: jks (required): Keystore format. Must be set to jks.

    • keystore_password (required): Password of the keystore file created in 2.

    • truststore_path (required): Location of the truststore file created in 2.

    • truststore_type: jks (required): Truststore format. Must be set to jks.

    • truststore_password (required): Password of the truststore file created in 2.

    • key_cache_millis (optional): Set to the interval at which DSE refreshes the key cache on the node in milliseconds. The default is 300000 (five minutes).

    • timeout (optional): Set to the socket timeout in milliseconds. The default is 1000.

    • protocol (optional): Set to the protocol for communicating between the node and KMIP key server. The default is TLS.

    • cipher_suites (optional): Set to a supported cipher for communicating between the node and KMIP key server. The default is JVM.

  4. Verify that the node can connect to the KMIP host by listing encryption keys on the remote KMIP server:

    dsetool managekmip list <kmip_group_name>

    dsetool picks up dse.yaml changes without requiring a restart.

    If problems connecting to the KMIP server occur, see Troubleshooting KMIP connections.

  5. Repeat these steps on all nodes in the cluster.

Encrypt configuration file properties

Configure DataStax Enterprise (DSE) to use a Key Management Interoperability Protocol (KMIP) encryption key to decrypt sensitive configuration properties. Use passwords encrypted with the KMIP key for the following properties:

  • dse.yaml LDAP properties:

    ldap_options.search_password
ldap_options.truststore_password

    Use plain text for the KMIP keystore/truststore passwords.

  • cassandra.yaml SSL properties:

    server_encryption_options.keystore_password
server_encryption_options.truststore_password
client_encryption_options.keystore_password
client_encryption_options.truststore_password

DSE caches encryption keys from the KMIP host and refreshes the cache at the interval set by the key_cache_millis property; the default setting is 5 minutes.

  1. Complete the steps in Adding a KMIP host.

    If any of the defined KMIP groups are not available, DSE startup fails.

  2. Back up the following configuration files:

    • cassandra.yaml: For package installations, this file is located at /etc/dse/cassandra/cassandra.yaml. For tarball installations, this file is located at INSTALL_DIRECTORY/resources/cassandra/conf/cassandra.yaml.

    • dse.yaml: For package installations, this file is located at /etc/dse/dse.yaml. For tarball installations, this file is located at INSTALL_DIRECTORY/resources/dse/conf/dse.yaml.

  3. Get the URL of the KMIP encryption key.

    • Create a new key

    • Use an existing key

    Create a new key and get the URL:

    dsetool createsystemkey '<AES/ECB/PKCS5>' <128> -k <kmip_group_name>

    Use the host_name and ID from the output to create the URL that you will enter in dse.yaml. For example, if the ID is 02-1655 and the host_name is kmip.example.com, then the URL is kmip://kmip.example.com/02-1655.

    To use an existing KMIP key, the URL syntax is kmip://<kmip_group_name>/<ID>.

    Look up the key ID:

    dsetool managekmip list <kmip_group_name>

    Use the host_name and ID from the output to create the URL that you will enter in dse.yaml. For example, if the ID is 02-449 and the host_name is kmip.example.com, then the URL is kmip://kmip.example.com/02-449.

    Result

    The URL of the second key in the following list is kmip://<host_name>/02-540.

    Keys on <host_name>:
    ID                                     Name               Cipher         State                Activation Date   Creation Date              Protect Stop Date   Namespace
02-449   82413ef3-4fa6-4d4d-9dc8-71370d731fe4_0        AES/CBC/PKCS5   Deactivated   Mon Apr 25 20:25:47 UTC 2016             n/a                            n/a         n/a
02-540   0eb2277e-0acc-4adb-9241-1dd84dde691c_0                  AES        Active   Tue May 31 12:57:59 UTC 2016             n/a                            n/a`

  4. Configure system property encryption settings in the dse.yaml:

    1. Enable system property encryption:

      config_encryption_active: true

    2. Set the URL of the KMIP key used to decrypt properties:

      config_encryption_key_name: <KMIP_key_URL>

      The <KMIP_key_URL> format is kmip://kmip_group_name/key-id. For example: kmip://<host_name>/02-1655.

  5. For each property, replace plain text passwords with encrypted passwords returned by running the dsetool encryptconfigvalue command:

    1. Encrypt the password:

      dsetool encryptconfigvalue
      Result 
      Using system key system_key

Enter value to encrypt:
Enter again to confirm:

Your encrypted value is:

+Vj5oHCR/jqfA+OJE2m8zA==

    2. Replace the old value with the returned encrypted value in the appropriate configuration file. For example, to replace the SSL truststore_password in the cassandra.yaml file:

      truststore_password: +Vj5oHCR/jqfA+OJE2m8zA==

      After the configuration file property encryption is enabled, DSE startup fails if any of the protected properties are not encrypted.

  6. Optional: Set up system resource encryption.

  7. Perform a rolling restart.

Encrypting system resources

Use a KMIP key to encrypt the system.batchlog and system.paxos tables, hint files and commit logs:

  1. Complete the steps in Adding a KMIP host.

    If any of the defined KMIP groups are not available, DSE startup fails.

  2. Edit the dse.yaml configuration file. For package installations, this file is located at /etc/dse/dse.yaml. For tarball installations, this file is located at INSTALL_DIRECTORY/resources/dse/conf/dse.yaml.

  3. In the dse.yaml file, configure encryption settings for system tables, the commit log, and the hints files:

    system_info_encryption:
  enabled: ( true | false )
  cipher_algorithm: <cipher_name>
  secret_key_strength: <length>
  key_provider: KmipKeyProviderFactory
  kmip_host: <kmip_group_name>
  chunk_length_kb: 64

    Set the following properties:

    • enabled (required): Set to true. On the next start up, system resources are encrypted. If the system tables have existing data, use nodetool upgradesstables to apply encryption.

    • key_provider (required): Set to KmipKeyProviderFactory.

    • kmip_host (required): Use the group name from the kmip_hosts section.

    • cipher_algorithm and secret_key_strength (optional): To ensure that KMIP generates a compatible key, configure the type of encryption key to use. When cipher_algorithm and secret_key_strength are set, DSE only uses a key that matches. If no matching key exists, start up fails. Set cipher_algorithm to the name of a supported JCE cipher algorithm to use, and then secret_key_strength to a compatible key length. DSE supports the following algorithms and lengths:

      Supported cipher algorithm names
      cipher_algorithm secret_key_strength

      AES

      128, 192, or 256

      DES

      56

      DESede

      112 or 168

      Blowfish

      32-448

      RC2

      40-128

    • chunk_length_kb (optional): Configures chunk size for SSTables. The default is 64.

  4. Perform a rolling restart.

  5. To encrypt existing data, run the following nodetool upgradesstables command on all nodes in the cluster:

    nodetool upgradesstables -a system batchlog paxos

Encrypt table data

Encrypt data stored in a table using a Key Management Interoperability Protocol (KMIP) key.

In DSE 6.8 or later, when you enable Transparent Data Encryption (TDE), all header data in indexes are encrypted, including partition keys in SSTable indexes. This feature is designed to protect sensitive data that might be present in the primary key. Consequently, DSE cannot access SSTables that are not decryptable.

When non-decryptable SSTables are present, DSE issues an error message during startup. If the error is ignored because the disk failure policy is specified as either ignore or best_effort, then DSE skips the non-decryptable SSTable and therefore ignores its content on queries without issuing a warning or error.

  1. Complete the steps in Adding a KMIP host.

    If any of the defined KMIP groups are not available, DSE startup fails.

  2. Locate the dse.yaml configuration file. The location of this file depends on the type of installation:

    • Package installations: /etc/dse/dse.yaml

    • Tarball installations: <installation_location>/resources/dse/conf/dse.yaml

  3. To create a new encrypted table using a key from a KMIP server:

    • Encryption without compression:

      CREATE TABLE customers
  ...
  WITH COMPRESSION =
  { 'class': 'Encryptor',
  'key_provider': 'KmipKeyProviderFactory',
  'kmip_host': 'kmip_group_name'
    ['key_keyspace': 'kmip_keyspace'],
  'cipher_algorithm': 'AES/ECB/PKCS5Padding',
  'secret_key_strength': 128 };

      • 'key_provider': 'KmipKeyProviderFactory' tells the encryptor to use a KMIP key server to manage its encryption keys. Include the 'key provider' entry only to specify to use a KMIP key server, otherwise omit this entry.

      • 'kmip_host': 'kmip_group_name' specifies the user-defined KMIP key server group name defined in the kmip_hosts section of the dse.yaml file.

      • 'kmip_host': 'kmip_group_name' ['key_keyspace': 'kmip_keyspace'] specify an optional KMIP keyspace. Use keyspaces to allow granular management of keys on a per table or keyspace basis.

    • Compression and encryption:

      CREATE TABLE customers
  ...
  WITH COMPRESSION =
  { 'class': 'EncryptingDeflateCompressor',
  'key_provider': 'KmipKeyProviderFactory',
  'kmip_host': 'kmip_group_name',
  'cipher_algorithm': 'AES/ECB/PKCS5Padding',
  'secret_key_strength': 128 };

  4. To encrypt a pre-existing table:

    1. Change the table compression settings:

      • Encryption without compression:

        ALTER TABLE customers
  ...
  WITH COMPRESSION =
  { 'class': 'Encryptor',
  'key_provider': 'KmipKeyProviderFactory',
  'kmip_host': 'kmip_group_name'
    ['key_keyspace': 'kmip_keyspace'],
  'cipher_algorithm': 'AES/ECB/PKCS5Padding',
  'secret_key_strength': 128 };

        • 'key_provider': 'KmipKeyProviderFactory' tells the encryptor to use a KMIP key server to manage its encryption keys. Include the 'key provider' entry only to specify to use a KMIP key server, otherwise omit this entry.

        • 'kmip_host': 'kmip_group_name' specifies the user-defined KMIP key server group name defined in the kmip_hosts section of the dse.yaml file.

        • ['key_keyspace': 'kmip_keyspace'] specify an optional KMIP keyspace. Use keyspaces to allow granular management of keys on a per table or keyspace basis.

      • Compression and encryption:

        ALTER TABLE customers
  ...
  WITH COMPRESSION =
  { 'class': 'EncryptingDeflateCompressor',
  'key_provider': 'KmipKeyProviderFactory',
  'kmip_host': 'kmip_group_name',
  'cipher_algorithm': 'AES/ECB/PKCS5Padding',
  'secret_key_strength': 128 };

    2. Encrypt existing data on all nodes in the cluster:

      nodetool upgradesstables -a [keyspace_name [table_name[ tablename]...]

Expire an encryption key

Security policies generally limit the amount of time an encryption key is in use. This section describes how to expire a key without re-encrypting the existing data. After a key expires, it is no longer used to encrypt new data, but it is still used to decrypt existing data.

To change the key used for both encryption and decryption, see Rekeying tables using a new key.

  1. Locate the dse.yaml configuration file. The location of this file depends on the type of installation:

    • Package installations: /etc/dse/dse.yaml

    • Tarball installations: <installation_location>/resources/dse/conf/dse.yaml

  2. Get a list of the available keys and states from the KMIP server:

    dsetool managekmip list kmip_group_name

    In this example result, a host named vormetricgroup has two keys:

    Keys on vormetricgroup:
    ID                                     Name               Cipher         State                Activation Date   Creation Date              Protect Stop Date   Keyspace
02-449   82413ef3-4fa6-4d4d-9dc8-71370d731fe4_0        AES/CBC/PKCS5   Deactivated   Mon Apr 25 20:25:47 UTC 2016             n/a                            n/a         n/a
02-540   0eb2277e-0acc-4adb-9241-1dd84dde691c_0                  AES        Active   Tue May 31 12:57:59 UTC 2016             n/a                            n/a

    DataStax Enterprise (DSE) supports one or more KMIP hosts. Each KMIP host is defined under a user-defined kmip_group_name in the kmip_hosts section of the dse.yaml.

  3. Expire the key:

    • Immediately expire the key:

      dsetool managekmip expirekey kmip\_group\_name key\_id

    • Schedule an expiration date:

      dsetool managekmip expirekey kmip\_group\_name key\_id datetime

      After the key expires, the database gets a new key for encryption the next time it refreshes the key cache (key_cache_millis). The default setting is 5 minutes. Expired keys are still available to decrypt data.

  4. Optional: Force a refresh of the DSE key cache by performing a rolling restart.

Rekey tables using a new key

Change the encryption key that is used for both encrypting new data and decrypting the existing data. Use these steps to secure the data after an event that potentially compromised an encryption key, such as a change in security administration staff. Before destroying the old key, revoke the compromised KMIP key, wait for the database key cache refresh, and then re-encrypt existing SSTables with the new key.

The database caches the encryption keys and refreshes the cache at an interval set by the key_cache_millis (default setting is 5 minutes). To get a new key, either wait for the key cache refresh interval or perform a rolling restart.

dsetool managekmip commands

The dsetool managekmip commands proxies KMIP commands to the corresponding host. The commands affect encryption keys as follows:

  • expirekey: Database stops using the key for encryption at the specified time and continues to use the expired key to decrypt existing data. Data re-keying is not required.

    Use this command to satisfy security policies that require periodic switching of the encryption key.

  • revoke: Permanently disables the key on the KMIP server. Database can no longer use the key for encryption, but continues to use the key for decryption of existing data. Re-encrypt existing data before completely removing the key from the KMIP server.

    Use this command as the first step when replacing a compromised key.

  • destroy: Completely removes the key from the KMIP server. Database can no longer use the key for encryption or decryption. Existing data that has not been re-encrypted becomes inaccessible.

    Use this command only after revoking a key and re-encrypting existing data.

Procedure

  1. Back up SSTables.

  2. Revoke the compromised key using the dsetool managekmip revoke:

    1. Get the ID of the KMIP encryption key you want to revoke from the KMIP server:

      dsetool managekmip list <kmip_groupname>

      The following is an example of a KMIP server that has two keys, active and deactivated.

      ID                                     Name          Cipher         State                Activation Date   Creation Date              Protect Stop Date   Keyspace
02-449   82413ef3-4fa6-4d4d-9dc8-71370d731fe4_0   AES/CBC/PKCS5   Deactivated   Mon Apr 25 20:25:47 UTC 2016             n/a                            n/a         n/a
02-540   0eb2277e-0acc-4adb-9241-1dd84dde691c_0             AES        Active   Tue May 31 12:57:59 UTC 2016             n/a                            n/a         n/a

      DataStax Enterprise (DSE) supports one or more KMIP hosts. Each KMIP host is defined under a user-defined <kmip_group_name> in the kmip_hosts section of the dse.yaml configuration file.

      The location of the dse.yaml file depends on the type of installation:

      • Package installations: /etc/dse/dse.yaml

      • Tarball installations: <installation_location>/resources/dse/conf/dse.yaml

    2. Revoke the key you want to replace using the ID:

      dsetool managekmip expirekey <kmip_groupname> <key_id>

      Revoking permanently deactivates the key on the KMIP server. When the key cache refreshes, a new key for encryption or decryption is automatically created. Revoked keys are used to decrypt existing data. DO NOT destroy the revoked key until after re-encrypting the existing data.

    3. Verify that the key State is Deactivated.

      dsetool managekmip list kmip_groupname

      The following is an example of a KMIP server that has two keys:

      ID                                     Name               Cipher         State                Activation Date   Creation Date              Protect Stop Date   Keyspace
02-449   82413ef3-4fa6-4d4d-9dc8-71370d731fe4_0        AES/CBC/PKCS5   Deactivated   Mon Apr 25 20:25:47 UTC 2016             n/a                            n/a         n/a
02-540   0eb2277e-0acc-4adb-9241-1dd84dde691c_0                  AES   Deactivated   Tue May 31 12:57:59 UTC 2016             n/a   Thu Jul 27 17:16:38 UTC 2017

  3. Refresh the database key cache using one of the following methods:

    KMIP keys are cached on the DSE node. DSE refreshes the cache and a new key is automatically generated by the KMIP server after key_cache_millis lapses; the default setting is 5 minutes.

  4. Optional: Get a list of the affected tables to re-encrypt using the new key using the DESC KEYSPACE command. For example to find all tables in the cycling keyspace that use the KMIP group:

    DESC KEYSPACE cycling

  5. Use nodetool upgradesstables to rewrite the encrypted SSTables using the new key. Run the relevant command on every node in the cluster:

    • Rewrite specific tables:

      nodetool upgradesstables --include-all-sstables keyspace_name table_name [table_name …]

    • Rewrite all tables in a specific keyspace:

      nodetool upgradesstables --include-all-sstables keyspace_name

    • Rewrite all keyspaces and tables:

      nodetool upgradesstables --include-all-sstables

  6. Optional: Remove the encryption key so that it is no longer available for decryption:

    dsetool managekmip destroy <key_id>

    The backed up SSTables are only accessible using the old key. Ensure that the data is accessible before removing the key.

Troubleshooting KMIP Connections

The following error message may occur if the DataStax Enterprise (DSE) SSL certificate is self-signed or from a CA that has not been added to the certificate chain on the KMIP server.

WARN 14:46:42,928 Unable to connect to KMIP host: 10.120.15.100:9005
com.cryptsoft.kmip.TTLVReadException: Error reading TTLV ResponseMessage. Got end-of-stream after reading 0 byte(s). Read bytes: (0) []

Ensure that trust between DSE and KMIP host has been properly configured, see Creating local SSL certificate and keystore files.

Was this helpful?

Give Feedback

How can we improve the documentation?

© Copyright IBM Corporation 2025 | Privacy policy | Terms of use |  Manage Privacy Choices

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: Contact IBM