Add customer keys
Astra DB allows you to associate your defined key from the cloud provider’s key management service (KMS) with a Customer Key in Astra DB.
Customer keys apply to only databases created after the customer key is configured. Customer keys cannot be applied to existing databases. Existing databases continue to use vendor-provided keys. |
This feature is available for paid DB Serverless users with AWS and Google Cloud regions, supported via Astra Portal and DevOps API. In the DevOps API, there are calls that allow you to programmatically manage customer keys:
-
Create a new association between a CMK (created in AWS KMS) or CMEK (created in Google Cloud KMS) and your Astra DB data, in a specific region.
-
List all Customer Keys that are associated with protecting the AWS- or Google Cloud-based Astra DB data for your organization.
-
List a specific Customer Key for an organization, based on the specified cloud provider (
aws
,gcp
) and region combination.
Prerequisites
-
An active Astra account
-
An active Astra DB database with the same cloud provider as your customer key
-
An application token for your database
-
The Organization Administrator role, Database Administrator role, or a custom role with the Read CMK Key and Write CMK Key organization permissions
-
Your organization ID, which you can get from any Astra Portal URL:
astra.datastax.com/org/ORGANIZATION_ID/
Set up a customer key
You can add only one customer key for each provider-region combination.
-
AWS KMS console
-
Google Cloud KMS console
-
Get your
provider_id
via REST API call:curl --request GET \ --url 'https://api.astra.datastax.com/v2/kms/provider/PROVIDER/region/REGION/accounts' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer APPLICATION_TOKEN'
Results
[{"organization_id":"ORGANIZATION_ID","provider_id":"PROVIDER_ID","provider":"aws"}]%
Copy the
provider_id
, which you’ll specify during your AWS KMS Customer Managed Key configuration. AWS uses thisprovider_id
to grant Astra DB permission to access this account. -
After authenticating into your AWS organization and account, navigate to the Key Management Service (KMS) page.
-
Click the left sidebar link for Customer managed keys.
-
Click Create key.
-
Choose the Symmetric option for a single key, which encrypts and decrypts data for your Astra DB database in an Amazon S3 bucket. Click Next.
-
On Add labels, enter a memorable alias for your CMK, such as
DataStax-byok-cmk
. Optionally, provide a brief Description and user-defined Tags. Click Next. -
On Define key administrative permissions, select your Key administrators from IAM users and roles. Select whether to Allow key administrators to delete this key. Click Next.
-
On Define key usage permissions, select your Key users from IAM users and roles, and optionally add Other AWS Accounts. Click Next.
-
On the Review page, review the selected options and generated Key policy.
-
Manually edit the Key policy to add this section with your
provider_id
.{ "Sid": "Allow an external account to use this KMS key", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::PROVIDER_ID:role/creator" }, "Action": [ "kms:EnableKey", "kms:Encrypt", "kms:Decrypt", "kms:GenerateDataKey", "kms:GenerateDataKeyWithoutPlaintext", "kms:ReEncryptTo", "kms:ReEncryptFrom", "kms:DescribeKey" ], "Resource": "*" },
-
Ensure that you have at least these actions defined with your ARN values:
-
Enable IAM User Permissions
-
Allow an external account to use this KMS key
-
Allow attachment of persistent resources
-
kms:CreateGrant
-
kms:ListGrants
-
kms:RevokeGrant
-
-
-
Click Finish.
Your new key is available on the Customer managed keys page.
On the AWS KMS Customer managed keys page, click your alias name. Copy the ARN of the key, which is the
keyID
that you’ll provide to Add Keys in the Astra Portal Security Settings or in the DevOps API/v2/kms
POST request payload.
-
Get your
provider_id
via REST API call:curl --request GET \ --url 'https://api.astra.datastax.com/v2/kms/provider/PROVIDER/region/REGION/accounts' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer APPLICATION_TOKEN'
Results
[{"organization_id":"ORGANIZATION_ID","provider_id":"PROVIDER_ID","provider":"gcp"}]%
Copy the 'provider_id', which you’ll specify during your Google Cloud Customer Managed Encryption Keys configuration. Google Cloud uses this
provider_id
to grant Astra DB permission to access this account. -
In the Google Cloud console, create a custom role and grant the minimum required permissions needed by Astra DB to access your Google Cloud Storage (GCS) buckets.
Custom roles let you group permissions and assign them to principals in your project or organization. You can manually select permissions or import permissions from another role.
-
After authenticating into your Google Cloud project, navigate to IAM & Admin.
-
On the IAM & Admin page, click Roles in the left-side navigation panel.
-
Click Create Role.
-
Enter your role Title and ID.
-
Click + ADD PERMISSIONS.
-
To narrow the results, select Cloud KMS Admin from Filter permissions by role.
-
Enable all of the KMS permissions for your custom role:
-
Click ADD.
-
Click CREATE.
You’ll specify this custom role during the CMEK steps outlined below.
-
-
Create a Customer Managed Encryption Key.
-
In your Google Cloud project, navigate to Key Management.
-
If KMS is not already enabled, click ENABLE on the Cloud KMS page.
-
Click + CREATE KEY RING.
Key rings group keys together to keep them organized. If you have an existing key ring, you can add your new key to the existing key ring.
-
Provide a name for your Key Ring.
-
For Location type, select Region. Select the same region of your Astra DB database.
-
Click CREATE.
-
On the Create Key page:
-
Enter a Key name.
-
For Protection level, select Software. Click Continue.
-
For Key material, select Generated Key. Click Continue.
-
For Purpose, choose Symmetric encrypt/decrypt. Click Continue.
-
Select your Key rotation period and start date to match your requirements. The default is 90 days. Click Continue.
-
Optionally, select any Additional settings you need.
-
Click CREATE.
-
-
On the Keys for YOUR_KEY_RING key ring page, click the link for your newly created key’s name.
-
On the Key: YOUR_KEY page, click the PERMISSIONS tab.
-
Click GRANT ACCESS.
-
In the Add principals box, enter these two principals with the
PROVIDER_ID
from your Astra DB Serverless.PROVIDER_ID-compute@developer.gserviceaccount.com service-PROVIDER_ID@gs-project-accounts.iam.gserviceaccount.com
-
In Assign roles, select the custom role you created.
-
Click SAVE.
-
On the Key: YOUR_KEY page, click the VERSIONS tab.
-
Under Actions, expand the three vertical dots and select Copy resource name.
The format of the copied resource name is:
projects/ID/locations/REGION_NAME/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME/cryptoKeyVersions/VERSION
Save the copied resource name of your Customer Managed Encryption Key. When adding your Customer Key, you will provide the resource name in Astra Portal.
-
Add the new customer key to Astra Portal
-
AWS
-
GCP
-
AWS with DevOps API
-
GCP with DevOps API
After completing the steps in AWS KMS to set up a CMK and granting Astra DB Serverless access to your S3 buckets, follow these steps in Astra Portal to add a new Customer Key:
-
From your left navigation, click Settings, and navigate to Security Settings.
-
On Security Settings, if your account includes the necessary permissions and roles, notice Key Encryption, a Premium feature.
-
In Key Encryption, click Add Keys.
-
In Add Customer Key:
-
Set Amazon Web Services as the cloud provider type.
-
Select the same region used by your AWS KMS Customer Managed Key and the Astra DB database that will have its data encrypted.
-
Paste in the Amazon Resource Name (ARN) value that you copied from the AWS KMS console, as outlined in the AWS KMS console section above.
-
Click Add Key.
-
-
When you’re ready, click Done.
After completing the steps in Google Cloud KMS to set up a CMEK and granting Astra DB Serverless access to your Google Cloud Storage, follow these steps in Astra Portal to add a new Customer Key:
-
From your left navigation, click Settings, and navigate to Security Settings.
-
On Security Settings, if your account includes the necessary permissions and roles, notice Key Encryption, a Premium feature.
-
In Key Encryption, click Add Keys.
-
In Add Customer Key:
-
Paste in the Google Cloud Key ID value that you copied from the Google Cloud KMW console, as outlined in the Google Cloud KMS console section above.
-
Click Add Key.
-
-
When you’re ready, click Done.
DevOps v2 API to manage Customer Keys
Once your CMK with assigned role and permissions are setup, you can use the DevOps API to submit calls that create and view associated Customer Keys for your Astra DB data.
Submit DevOps API calls to create and view Customer Keys in your Astra DB organization.
If you are using Postman for your API calls, ensure you use the raw
option to enter the body of your API call.
For descriptions of the /v2/kms
calls, see the sections below. For more, see the DevOps v2 API reference details.
Create a Customer Key for Astra DB
Use the following POST
to create a Customer Key in Astra DB that will be associated with an existing Customer Managed Key in the AWS Key Management Service (KMS).
In the request payload, specify your Astra DB organizationID
, and the keyID
of the Customer Managed Key that you created in AWS KMS, and the region
.
In each Astra DB organization, there can be one Customer Key for a given cloud provider and region combination.
A successful Customer Key creation returns 200 OK
in the response.
curl --request POST \
--url 'https://api.astra.datastax.com/v2/kms' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer APPLICATION_TOKEN' \
--data '{
"orgId": "ORGANIZATION_ID",
"aws": {
"keyID": "ARN",
"region": "REGION"
}
}'
Results
When the key is found for the given provider and region, the 200 OK response contains:
{
"orgId": "ORGANIZATION_ID",
"aws":
{
"keyID": "PROVIDER_ACCOUNT_ID",
"region": "REGION"
}
}
Other potential response codes:
-
400 Bad request.
-
403 The user is forbidden to perform the operation.
-
500 A server error occurred.
{
"errors": [
{
"description": "STRING",
"internalCode": "STRING",
"internalTxId": "STRING"
}
]
}
Get all Customer Keys in an Astra DB organization
Retrieve all existing Customer Keys of an Astra DB organization.
curl --request GET \
--url 'https://api.astra.datastax.com/v2/kms' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer APPLICATION_TOKEN'
Results
When one or more keys are found for the organization, the 200 OK response contains, for each key:
[
{
"OrganizationID": "ORGANIZATION_ID",
"CloudProvider": "aws",
{
"keyID": "ARN",
"region": "REGION"
}
}
]
If there are no keys setup for the organization, the 404 response contains:
{
"description": "Organization: ORGANIZATION_ID does not have
any CMKs associated with it."
}
Other potential response codes:
-
400 Bad request.
-
403 The user is forbidden to perform the operation.
-
500 A server error occurred.
Get a single Customer Key in an Astra DB organization
Retrieves a Customer Key in an Astra DB organization for a given cloud provider and region combination. The provider value in this case should be aws
.
curl --request GET \
--url 'https://api.astra.datastax.com/v2/kms/provider/aws/region/REGION/accounts' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer APPLICATION_TOKEN'
Results
When the key is found for the given provider and region, the 200 OK response contains:
{
"orgId": "ORGANIZATION_ID",
"provider": "aws":
{
"keyID": "PROVIDER_ACCOUNT_ID",
"region": "REGION"
}
}
If there are no keys setup for the organization’s provider and region combination, the 404 response contains:
{
"description": "Could not find PROVIDER data in
region REGION for organization ORGANIZATION_ID."
}
Other potential response codes:
-
400 Bad request.
-
403 The user is forbidden to perform the operation.
-
500 A server error occurred.
Get an array of Astra DB cloud provider accounts
Retrieves a Customer Key in an Astra DB organization for a given cloud provider and region combination.
The provider value in this case should be aws
.
The GET response also returns the provider ID of the storage account where your Amazon S3 buckets exist. If not done already, you can use this information to configure or modify (back in the AWS Key Management Service) your KMS Customer Managed Key, so that Astra DB has access to the key.
curl --request GET \
--url 'https://api.astra.datastax.com/v2/kms/provider/aws/region/REGION/accounts' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer APPLICATION_TOKEN'
Results
When the key is found for the given provider and region, the 200 OK response contains an array of:
[
{
"organization_id": "ORGANIZATION_ID",
"provider_id": "PROVIDER_ACCOUNT_ID",
"provider": "aws"
}
]
If there are no keys setup for the organization’s provider and region combination, the 404 response contains:
{
"description": "Could not find PROVIDER data in
region REGION for organization ORGANIZATION_ID."
}
Other potential response codes:
-
400 Bad request.
-
403 The user is forbidden to perform the operation.
-
500 A server error occurred.
DevOps v2 API to manage Customer Keys
Once your CMEK with assigned role and permissions are setup, you can use the DevOps API to submit calls that create and view associated Customer Keys for your Astra DB data.
Submit DevOps API calls to create and view Customer Keys in your Astra DB organization.
If you are using Postman for your API calls, ensure you use the raw
option to enter the body of your API call.
For descriptions of the /v2/kms
calls, see the sections below. For more, see the DevOps API reference.
Create a Customer Key for Astra DB
Use the following POST
to create a Customer Key in Astra DB that will be associated with an existing Customer Managed Encryption Key (CMEK) that was created in Google Cloud.
In the request payload, specify your Astra DB orgID
, and the keyID
of the Customer Managed Encrypted Key that you created in Google Cloud KMS, and the region
.
The keyID
for the POST payload is the CMEK’s resource name, as generated in the Google Cloud console.
In each Astra DB organization, there can be one Customer Key for a given cloud provider and region combination.
A successful Customer Key creation returns 200 OK
in the response.
curl --request POST \
--url 'https://api.astra.datastax.com/v2/kms' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer APPLICATION_TOKEN' \
--data '{
"orgId": "ORGANIZATION_ID",
"gcp": {
"keyID": "RESOURCE_NAME",
"region": "REGION"
}
}'
Results
200 OK
Other potential response codes:
-
400 Bad request.
-
403 The user is forbidden to perform the operation.
-
500 A server error occurred.
{
"errors": [
{
"description": "STRING",
"internalCode": "STRING",
"internalTxId": "STRING"
}
]
}
Get all Customer Keys in an Astra DB organization
Retrieve all existing Customer Keys of an Astra DB organization.
curl --request GET \
--url 'https://api.astra.datastax.com/v2/kms/' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer APPLICATION_TOKEN'
Results
When one or more keys are found for the organization, the 200 OK response contains, for each key:
[
{
"OrganizationID": "ORGANIZATION_ID",
"CloudProvider": "gcp",
"KeyID": "CUSTOMER_KEY_ID_STRING",
"Region": "REGION"
}
]
If there are no keys setup for the organization, the 404 response contains:
{
"description": "Organization: ORGANIZATION_ID does not have
any CMKs associated with it."
}
Other potential response codes:
-
400 Bad request.
-
403 The user is forbidden to perform the operation.
-
500 A server error occurred.
Get a single Customer Key in an Astra DB organization
Retrieves a Customer Key in an Astra DB organization for a given cloud provider & region combination. The provider value in this example should be gcp
.
The GET response also returns the provider ID of the storage account where your Google Cloud Storage (GCS) buckets exist. If not done already, you can use this information to configure or modify (back in the Google Cloud console) your CMEK, so that Astra DB has access to the key.
curl --request GET \
--url 'https://api.astra.datastax.com/v2/kms/provider/PROVIDER/region/REGION/accounts' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer APPLICATION_TOKEN'
Results
When the key is found for the given provider and region, the 200 OK response contains:
{
"orgId": "ORGANIZATION_ID",
"provider": "AWS | GCP",
{
"keyID": "PROVIDER_ACCOUNT_ID",
"region": "REGION"
}
}
If there are no keys setup for the organization’s provider and region combination, the 404 response contains:
{
"description": "Could not find PROVIDER data in
region REGION for organization ORGANIZATION_ID."
}
Other potential response codes:
-
400 Bad request.
-
403 The user is forbidden to perform the operation.
-
500 A server error occurred.
The Security Settings page displays any defined Customer Keys in your organization.
These are the possible statuses:
- In-Use
-
A database in your organization is actively using the key.
- Available
-
Indicates a Customer Key has been added for your organization, but no Astra DB Serverless database with that Provider and Region combination is using the key.
Once you add your customer keys, your Astra DB databases in the organization with the same provider and region combination are encrypted via the cloud provider’s customer key in the storage buckets.