Add customer keys

Astra DB allows you to associate your defined key from the cloud provider’s key management service (KMS) with a Customer Key in Astra DB.

This feature is available for paid DB Serverless users with AWS and Google Cloud regions, supported via Astra Portal and DevOps API. In the DevOps API, there are calls that allow you to programmatically manage customer keys:

Create a new association between a CMK (created in AWS KMS) and your Astra DB data, in a specific region.
List all Customer Keys that are associated with protecting the AWS-based Astra DB data for your organization.
List a specific Customer Key for an organization, based on the specified cloud provider (aws) & region combination.

Prerequisites

You have an active Astra account.
You have the Organization Administrator or Database Administrator role, or a custom role with the org-cmk-read and org-cmk-write permissions enabled.
Create an Astra DB database using the Astra Portal based with the cloud provider you will create a customer key.
Your application token, which you can generate from your database Overview in the Astra Portal.
Ensure that you know your Astra Portal organization ID. When you log into the Astra DB console, your organization ID is in the generated URL. For example, in the URL https://astra.DataStax.com/org/a99999c7-b934-436c-9999-9999999a3b5d, the organization ID is a99999c7-b934-436c-9999-9999999a3b5d.

Set up a customer key

You can add only one customer key for each provider-region combination.

AWS KMS console
Google Cloud KMS console

Get your provider_id via REST API call:

curl --request GET \
 --url 'https://api.astra.DataStax.com/v2/kms/provider/PROVIDER/region/REGION/accounts' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer APPLICATION_TOKEN'

Results

[{"organization_id":"<organization_id>","provider_id":"<provider_id>","provider":"aws"}]%

Copy the 'provider_id', which you’ll specify during your AWS KMS Customer Managed Key configuration. AWS uses this provider_id to grant Astra DB permission to access this account.

After authenticating into your AWS organization and account, navigate to the Key Management Service (KMS) page.
Click the left sidebar link for Customer managed keys.
Click Create key.
Choose the Symmetric option for a single key, which encrypts and decrypts data for your Astra DB database in an Amazon S3 bucket. Click Next.
On Add labels, enter a memorable alias for your CMK, such as DataStax-byok-cmk. Optionally, provide a brief Description and user-defined Tags. Click Next.
On Define key administrative permissions, select your Key administrators from IAM users and roles. Select whether to Allow key administrators to delete this key. Click Next.
On Define key usage permissions, select your Key users from IAM users and roles, and optionally add Other AWS Accounts. Click Next.
On the Review page, review the selected options and generated Key policy.

Manually edit the Key policy to add this section with your provider_id.

        {
            "Sid": "Allow an external account to use this KMS key",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<provider_id>:role/creator"
            },
            "Action": [
                "kms:EnableKey",
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:GenerateDataKey",
                "kms:GenerateDataKeyWithoutPlaintext",
                "kms:ReEncryptTo",
                "kms:ReEncryptFrom",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        },

Ensure that you have at least these actions defined with your ARN values:
- Enable IAM User Permissions
- Allow an external account to use this KMS key
- Allow attachment of persistent resources
  - kms:CreateGrant
  - kms:ListGrants
  - kms:RevokeGrant

Click Finish.

Your new key is available on the Customer managed keys page.

On the AWS KMS Customer managed keys page, you click the alias name you defined. Copy the ARN of the key, which is the keyID that you’ll provide to Add Keys in the Astra Portal Security Settings or in the DevOps API /v2/kms POST request payload.

Get your provider_id via REST API call:

curl --request GET \
 --url 'https://api.astra.DataStax.com/v2/kms/provider/PROVIDER/region/REGION/accounts' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer APPLICATION_TOKEN'

Results

[{"organization_id":"<organization_id>","provider_id":"<provider_id>","provider":"aws"}]%

Copy the 'provider_id', which you’ll specify during your Google Cloud Customer Managed Encryption Keys configuration. Google Cloud uses this provider_id to grant Astra DB permission to access this account.

In the Google Cloud console, create a custom role and grant the minimum required permissions needed by Astra DB to access your Google Cloud Storage (GCS) buckets.

Custom roles let you group permissions and assign them to principals in your project or organization. You can manually select permissions or import permissions from another role.
1. After authenticating into your Google Cloud project, navigate to IAM & Admin.
2. On the IAM & Admin page, click Roles in the left-side navigation panel.
3. Click Create Role.
4. Enter your role Title and ID.
5. Click + ADD PERMISSIONS.
6. To narrow the results, select Cloud KMS Viewer and Cloud KMS CryptoKey Encrypter/decrypter from Filter permissions by role.
7. At a minimum, enable the following permissions for your custom role:
  cloudkms.cryptoKeys.get cloudkms.cryptoKeyVersions.useToDecrypt cloudkms.cryptoKeyVersions.useToEncrypt cloudkms.locations.get cloudkms.locations.list resourcemanager.projects.get
8. Click ADD.
9. Click CREATE.
  
  You’ll specify this custom role during the CMEK steps outlined below.
Create a Customer Managed Encryption Key.
1. In your Google Cloud project, navigate to Key Management.
2. If KMS is not already enabled, click ENABLE on the Cloud KMS page.
3. Click + CREATE KEY RING.
  
  Key rings group keys together to keep them organized. If you have an existing key ring, you can add your new key to the existing key ring.
4. Provide a name for your Key Ring.
5. For Location type, choose Region. Select the same region of your Astra DB database.
6. Click CREATE.
7. On the Create Key page:
  1. Enter a Key name.
  2. For Protection level, select Software. Click Continue.
  3. For Key material, select Generated Key. Click Continue.
  4. For Purpose, choose Symmetric encrypt/decrypt. Click Continue.
  5. Select your Key rotation period and start date to match your requirements. The default is 90 days. Click Continue.
  6. Optionally, select any Additional settings you need.
  7. Click CREATE.
8. On the Keys for YOUR_KEY_RING key ring page, click the link for your newly created key’s name.
9. On the Key: YOUR_KEY page, click the PERMISSIONS tab.
10. Click GRANT ACCESS.
11. In the Add principals box, enter these two principals with the PROVIDER_ID from your Astra DB Serverless.
  *PROVIDER_ID*-compute@developer.gserviceaccount.com service-*PROVIDER_ID*@gs-project-accounts.iam.gserviceaccount.com
  See [_prerequisite_api_step_to_determine_account_for_gcs_buckets]).
12. In Assign roles, select the custom role you created.
13. Click SAVE.
14. On the Key: YOUR_KEY page, click the VERSIONS tab.
15. Under Actions, expand the three vertical dots and select Copy resource name.
  
  The format of the copied resource name is:
  
  projects/ID/locations/REGION_NAME/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME/cryptoKeyVersions/VERSION
  
  Save the copied resource name of your Customer Managed Encryption Key. When adding your Customer Key, you will provide the resource name in Astra Portal.

Add the new customer key to Astra Portal

AWS
GCP
AWS with DevOps API
GCP with DevOps API

After completing the steps in AWS KMS to set up a CMK and granting Astra DB Serverless access to your S3 buckets, follow these steps in Astra Portal to add a new Customer Key:

From your left navigation, click Settings, and navigate to Security Settings.
On Security Settings, if your account includes the necessary permissions and roles, notice Key Encryption, a Premium feature.
In Key Encryption, click Add Keys.
In Add Customer Key:
1. Set Amazon Web Services as the cloud provider type.
2. Select the same region used by your AWS KMS Customer Managed Key and the Astra DB database that will have its data encrypted.
3. Paste in the Amazon Resource Name (ARN) value that you copied from the AWS KMS console, as outlined in the AWS KMS console section above.
4. Click Add Key.
When you’re ready, click Done.

After completing the steps in Google Cloud KMS to set up a CMEK and granting Astra DB Serverless access to your Google Cloud Storage, follow these steps in Astra Portal to add a new Customer Key:

From your left navigation, click Settings, and navigate to Security Settings.
On Security Settings, if your account includes the necessary permissions and roles, notice Key Encryption, a Premium feature.
In Key Encryption, click Add Keys.
In Add Customer Key:
1. Paste in the Google Cloud Key ID value that you copied from the Google Cloud KMW console, as outlined in the Google Cloud KMS console section above.
2. Click Add Key.
When you’re ready, click Done.

DevOps v2 API to manage Customer Keys

Once your CMK with assigned role and permissions are setup, you can use the DevOps API to submit calls that create and view associated Customer Keys for your Astra DB data. Submit DevOps API calls to create and view Customer Keys in your Astra DB organization. If you are using Postman for your API calls, ensure you use the raw option to enter the body of your API call.

For descriptions of the /v2/kms calls, see the sections below. For more, see the DevOps v2 API reference details.

Create a Customer Key for Astra DB

Use the following POST to create a Customer Key in Astra DB that will be associated with an existing Customer Managed Key in the AWS Key Management Service (KMS).

In the request payload, specify your Astra DB organizationID, and the keyID of the Customer Managed Key that you created in AWS KMS, and the region. In each Astra DB organization, there can be one Customer Key for a given cloud provider and region combination. A successful Customer Key creation returns 200 OK in the response.

curl --request POST \
  --url 'https://api.astra.DataStax.com/v2/kms' \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer APPLICATION_TOKEN' \
  --data '{
  "orgId": "ORGANIZATION_ID",
  "aws": {
    "keyID": "ARN",
    "region": "REGION"
  }
}'

Results

When the key is found for the given provider and region, the 200 OK response contains:

{
  "orgId": "The ID of your organization",
  "aws":
  {
    "keyID": "The ID of the provider account where DataStax stores your data buckets",
    "region": "region-name"
  }
}

Other potential response codes:

400 Bad request.
403 The user is forbidden to perform the operation.
500 A server error occurred.

{
  "errors": [
    {
      "description": "STRING",
      "internalCode": "STRING",
      "internalTxId": "STRING"
    }
  ]
}

Get all Customer Keys in an Astra DB organization

Retrieve all existing Customer Keys of an Astra DB organization.

curl --request GET \
 --url 'https://api.astra.DataStax.com/v2/kms' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer APPLICATION_TOKEN'

Results

When one or more keys are found for the organization, the 200 OK response contains, for each key:

[
  {
    "OrganizationID": "*ORGANIZATION_ID*",
    "CloudProvider": "aws",
    {
      "keyID": "ARN",
      "region": "REGION"
    }
  }
]

If there are no keys setup for the organization, the 404 response contains:

{
  "description": "Organization: ORGANIZATION_ID does not have
                  any CMKs associated with it."
}

Other potential response codes:

400 Bad request.
403 The user is forbidden to perform the operation.
500 A server error occurred.

Get a single Customer Key in an Astra DB organization

Retrieves a Customer Key in an Astra DB organization for a given cloud provider & region combination. The provider value in this case should be aws.

curl --request GET \
 --url 'https://api.astra.DataStax.com/v2/kms/provider/aws/region/REGION/accounts' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer APPLICATION_TOKEN'

Results

When the key is found for the given provider and region, the 200 OK response contains:

{
  "orgId": "The ID of your organization",
  "provider": "aws":
  {
    "keyID": "The ID of the provider account where DataStax stores your data buckets",
    "region": "region-name"
  }
}

If there are no keys setup for the organization’s provider and region combination, the 404 response contains:

{
  "description": "Could not find PROVIDER data in
                  region REGION for organization ORGANIZATION_ID."
}

Other potential response codes:

400 Bad request.
403 The user is forbidden to perform the operation.
500 A server error occurred.

Get an array of Astra DB cloud provider accounts

Retrieves a Customer Key in an Astra DB organization for a given cloud provider and region combination. The provider value in this case should be aws.

The GET response also returns the provider ID of the storage account where your Amazon S3 buckets exist. If not done already, you can use this information to configure or modify (back in the AWS Key Management Service) your KMS Customer Managed Key, so that Astra DB has access to the key.

curl --request GET \
 --url 'https://api.astra.DataStax.com/v2/kms/provider/aws/region/REGION/accounts' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer APPLICATION_TOKEN'

Results

When the key is found for the given provider and region, the 200 OK response contains an array of:

[
  {
    "organization_id": "ID of your organization",
    "provider_id": "ID of the provider account
                   where DataStax stores your data buckets",
    "provider": "aws"
  }
]

If there are no keys setup for the organization’s provider and region combination, the 404 response contains:

{
  "description": "Could not find PROVIDER data in
                  region REGION for organization ORGANIZATION_ID."
}

Other potential response codes:

400 Bad request.
403 The user is forbidden to perform the operation.
500 A server error occurred.

DevOps v2 API to manage Customer Keys

Once your CMEK with assigned role and permissions are setup, you can use the DevOps API to submit calls that create and view associated Customer Keys for your Astra DB data. Submit DevOps API calls to create and view Customer Keys in your Astra DB organization. If you are using Postman for your API calls, ensure you use the raw option to enter the body of your API call.

For descriptions of the /v2/kms calls, see the sections below. For more, see the DevOps v2 API reference details.

Create a Customer Key for Astra DB

Use the following POST to create a Customer Key in Astra DB that will be associated with an existing Customer Managed Encryption Key (CMEK) that was created in Google Cloud.

In the request payload, specify your Astra DB orgID, and the keyID of the Customer Managed Encrypted Key that you created in Google Cloud KMS, and the region. The keyID for the POST payload is the CMEK’s resource name, as generated in the Google Cloud console. In each Astra DB organization, there can be one Customer Key for a given cloud provider and region combination. A successful Customer Key creation returns 200 OK in the response.

curl --request POST \
  --url 'https://api.astra.DataStax.com/v2/kms' \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer APPLICATION_TOKEN' \
  --data '{
  "orgId": "ORGANIZATION_ID",
  "gcp": {
    "keyID": "*RESOURCE_NAME*",
    "region": "*REGION*"
  }
}'

Results

200 OK

Other potential response codes:

400 Bad request.
403 The user is forbidden to perform the operation.
500 A server error occurred.

{
  "errors": [
    {
      "description": "STRING",
      "internalCode": "STRING",
      "internalTxId": "STRING"
    }
  ]
}

Get all Customer Keys in an Astra DB organization

Retrieve all existing Customer Keys of an Astra DB organization.

curl --request GET \
 --url 'https://api.astra.DataStax.com/v2/kms/' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer APPLICATION_TOKEN'

Results

When one or more keys are found for the organization, the 200 OK response contains, for each key:

[
  {
    "OrganizationID": ""ORGANIZATION_ID"",
    "CloudProvider": "gcp",
    "KeyID": "String - The ID of the Customer Key",
    "Region": "REGION"
  }
]

If there are no keys setup for the organization, the 404 response contains:

{
  "description": "Organization: ORGANIZATION_ID does not have
                  any CMKs associated with it."
}

Other potential response codes:

400 Bad request.
403 The user is forbidden to perform the operation.
500 A server error occurred.

Get a single Customer Key in an Astra DB organization

Retrieves a Customer Key in an Astra DB organization for a given cloud provider & region combination. The provider value in this example should be gcp.

The GET response also returns the provider ID of the storage account where your Google Cloud Storage (GCS) buckets exist. If not done already, you can use this information to configure or modify (back in the Google Cloud console) your CMEK, so that Astra DB has access to the key.

curl --request GET \
 --url 'https://api.astra.DataStax.com/v2/kms/provider/PROVIDER/region/REGION/accounts' \
 --header 'Accept: application/json' \
 --header 'Authorization: Bearer APPLICATION_TOKEN'

Results

When the key is found for the given provider and region, the 200 OK response contains:

{
  "orgId": "The ID of your organization",
  "provider": "aws or gcp":
  {
    "keyID": "The ID of the provider account where DataStax stores your data buckets",
    "region": "region-name"
  }
}

If there are no keys setup for the organization’s provider and region combination, the 404 response contains:

{
  "description": "Could not find PROVIDER data in
                  region REGION for organization ORGANIZATION_ID."
}

Other potential response codes:

400 Bad request.
403 The user is forbidden to perform the operation.
500 A server error occurred.

The Security Settings page displays any defined Customer Keys in your organization.

Security Settings displays keys for your organization.

These are the possible statuses:

In-Use: A database in your organization is actively using the key.
Available: Indicates a Customer Key has been added for your organization, but no Astra DB Serverless database with that Provider and Region combination is using the key.

Once you add your customer keys, your Astra DB databases in the organization with the same provider and region combination are encrypted via the cloud provider’s customer key in the storage buckets.

Add customer keys

Prerequisites

Set up a customer key

Add the new customer key to Astra Portal

Was this helpful?

Give Feedback

Add customer keys

Prerequisites

Set up a customer key

Add the new customer key to Astra Portal

Related topics

Was this helpful?