BYOK GCP with Astra Portal (Serverless)
Encryption is a widely accepted mechanism to secure data against breaches. By default, DataStax Astra DB encrypts data, and providers such as Google Cloud offer encryption solutions. However, you may want to further limit data access, because cloud providers have access to the keys and ultimately to the data.
To address this security concern, DataStax Astra DB allows you to associate a Customer Managed Encryption Key (one per region) that you defined in Google Cloud console with a Customer Key that you create in Astra DB.
In this topic, we’ll use these terms:
-
Bring Your Own Key (BYOK) refers to the overall capability.
-
Customer Managed Encryption Key (CMEK) refers to a particular key type in Google Cloud Key Management Service (KMS).
-
Customer Key refers to the corresponding key association in Astra DB.
This BYOK feature:
|
For related details, see the Customer Keys API reference.
BYOK for Astra DB Classic is available on AWS only, via DevOps API, and upon request. Not configurable via Astra Portal.
Introduction
Data encryption is defined as a process that transforms data into an encoded format. Once encoded, the data is incomprehensible without being decrypted. Data encryption is essential for organizations in all industries because it protects data from unauthorized access. When thinking of data encryption, two main scenarios are often considered:
-
Data at rest
Encrypting data while it is stored in the file storage in use.
-
Data in transit
Encrypting data while it travels through private or public networks.
BYOK allows customers to manage encryption for data at rest.
Benefits
With BYOK, you can take full control of the encryption keys when storing data in the cloud. Google Cloud Key Management Service (KMS) provides protection against data breaches by alerting you when tampering occurs. In KMS, you can configure specific policies to adhere to compliance guidelines, such as auditing, key rotation, and access.
Setting up a corresponding Customer Key for a Google Cloud based Astra DB database separates the:
-
The encrypted lock
-
The key that encrypts/decrypts data
This separation of lock and key is considered a best practice to secure data via encryption.
After setting up a Customer Managed Encryption Key (CMEK) in your Google Cloud project, you can use either Astra Portal or the DataStax DevOps API, to associate the existing CMEK with a Customer Key in Astra DB.
In Astra Portal, under Settings, see the Key Encryption section of Security Settings.
In addition to the console UI, BYOK provides three DataStax DevOps API calls that allow you to programmatically:
-
Create a new association between a CMEK (created in Google Cloud KMS) and your Astra DB data, in a specific region.
-
List all Customer Keys that are associated with protecting the Google Cloud based Astra DB data for your organization.
-
List a specific Customer Key for an organization, based on the specified cloud provider (
gcp
) & region combination.
Key deletion
Please contact DataStax Support if you need to delete a key from your Astra DB organization. If you agree, the DataStax Support team may perform the key deletion on your behalf. Once a registered association with a Google Cloud Customer Managed Encryption Key is deleted from your Astra DB organization, the default data encryption provided by Astra DB is used.
Prerequisites
-
Create your Astra DB database using Astra Portal. In the case of this BYOK feature, create a Google Cloud based database, and choose one of the available regions to start.
-
Set up a Customer Managed Encryption Key (one per region, as needed) in Google Cloud KMS.
-
Ensure you have the required Roles and permissions.
-
Ensure that you know your Astra DB organization ID. When you log into the Astra DB console, your organization ID is in the generated URL. For example, in the URL https://astra.datastax.com/org/a99999c7-b934-436c-9999-9999999a3b5d/manage, the organization ID is
a99999c7-b934-436c-9999-9999999a3b5d
.
Multi-region support
The BYOK feature is supported in multi-region Astra DB environments; however, each region is encrypted using its own key. Keys cannot be shared across regions. For a given organization:
|
Pricing
There is no additional cost to using BYOK with Astra DB. As noted previously, BYOK is not available with the Astra DB Free Plan.
Customer Managed Encryption Keys in Google Cloud may incur a monthly fee, and a fee for use in excess of the Google Cloud free tier. The fees are counted against the Google Cloud KMS quotas for your project. For details, see Customer Managed Encryption Key in the Google Cloud documentation.
Roles and permissions
The following Astra DB roles can manage Customer Keys.
-
Organization Administrator
-
Database Administrator
To manage Customer Keys, your Astra DB account must have these permissions enabled.
-
org-cmk-read
-
org-cmk-write
Google Cloud KMS
This section describes steps you’ll perform in Google Cloud console to:
-
Create a custom role with specific permissions needed by Astra DB
-
Create a Customer Managed Encryption Key
Once your CMEK with assigned role and permissions are setup, you can use the DevOps API to submit calls that create and view associated Customer Keys for your Astra DB data.
Prerequisite API step to determine account for GCS buckets
Before you create a custom role and a Customer Managed Encryption Key (CMEK) in Google Cloud console, determine (for a given region) in which project does Astra DB store your Google Cloud Storage (GCS) buckets. To find that information, use cURl or Postman to submit a
The GET’s response includes the ID of the Google Cloud Storage (GCS) account, into which Astra DB will store your database’s encrypted data. Copy the ID number returned in the GET’s response. Then, when you configure your Google Cloud CMEK (see the steps below), you’ll specify this account number, and grant DataStax permission to access this account. |
Create a custom role and grant permissions
In Google Cloud console, create a custom role and grant the minimum required permissions needed by Astra DB to access your Google Cloud Storage (GCS) buckets. Custom roles let you group permissions and assign them to principals in your project or organization. You can manually select permissions or import permissions from another role.
-
After authenticating into your Google Cloud project, search for "IAM & Admin".
-
On the "IAM & Admin" page, from the left-side navigation panel, choose Roles.
-
Click Create Role.
-
Click ADD PERMISSIONS and filter on
kms
. -
From among the filtered results, further narrow down by enabling Cloud KMS Viewer and Cloud KMS CryptoKey Encrypter/decrypter.
-
Scroll down the refreshed results list, and at a minimum, enable the following permissions for your custom role:
cloudkms.cryptoKeyVersions.useToDecrypt cloudkms.cryptoKeyVersions.useToEncrypt cloudkms.cryptoKeys.get
Example:
-
Click ADD.
-
Modify values such as the custom role’s Title and ID string.
-
Click CREATE.
You’ll specify this custom role during the CMEK steps outlined below. |
Create a Customer Managed Encryption Key
-
While authenticated into your Google Cloud project, search for, or navigate to, the "Key Management" service. The Cloud Key Management Service (Cloud KMS) lets you create, use, rotate, and manage cryptographic keys. A cryptographic key is a resource that is used for encrypting and decrypting data or for producing and verifying digital signatures.
-
On the Cloud KMS page, if you haven’t already, click ENABLE.
-
Click Create Key Ring, or open an existing Key Ring if already defined. Key rings group keys together to keep them organized.
-
If new, provide a name for your Key Ring, such as
testkeyring
. -
For Location type, choose Region.
-
Select the same region of your Astra DB database(s).
-
-
On the "Create Key" page:
-
Enter a Name for your key.
-
For Key type, choose Generated Key.
-
For Protection level, choose Software.
-
For Purpose, choose Symmetric encrypt/decrypt.
-
Decide on the key Rotation cadence to match your preference. The default is 90 days.
-
When ready, click CREATE.
-
-
On the "Keys for <your-key-ring-name> key ring" page, click the link for your newly created key’s name.
-
On the "Key: <your-key-name>" page, click the PERMISSIONS tab.
-
In this step, you’ll provide:
-
The storage account ID returned in the GET
/v2/kms/provider/gcp/region/<region-name>/accounts
API call. See Prerequisite API step to determine account for GCS buckets. -
The custom role you created earlier. See Create a custom role and grant permissions.
Provide those values so you can properly setup, with the necessary role and permissions, the following principles:
<projectNumber>-compute@developer.gserviceaccount.com service-<projectNumber>@gs-project-accounts.iam.gserviceaccount.com
Where
<projectNumber>
is the output of GET/v2/kms/provider/gcp/region/<region-name>/accounts
.-
On this PERMISSIONS tab, click ADD.
-
Paste into the New principals textbox the account ID that you copied from the response of the GET
/v2/kms/provider/gcp/region/<region-name>/accounts
. (Again, refer to Prerequisite API step to determine account for GCS buckets). -
Select the name of the custom role that you created earlier. As a reminder, in an earlier step, we applied the following permissions to the custom role:
cloudkms.cryptoKeyVersions.useToDecrypt cloudkms.cryptoKeyVersions.useToEncrypt cloudkms.cryptoKeys.get
Those are the minimum permissions needed in the role that’s assigned to your CMEK.
-
When you’re ready, click SAVE.
-
-
-
On the "Key: <your-key-name>" page, click the VERSIONS tab.
-
Under Actions, expand the three vertical dots and select Copy resource name. Example:
The format of the copied resource name is:
projects/<id>/locations/<region-name>/keyRings/<your-key-ring-name>/cryptoKeys/<your-key-name>/cryptoKeyVersions/<version>
Optionally, you can remove the
/cryptoKeyVersions/<version>
portion at the end of the copied resource name. Copy the rest. Example:projects/this-that-999999/locations/us-east1/keyRings/testkeyring/cryptoKeys/mydefaultkey
Save the copied resource name of your Customer Managed Encryption Key. You’ll provide its value while adding a Customer Key association in Astra Portal. For those steps, see the next section. |
Astra DB console’s Security Settings & Key Encryption
Reminder: By default, Astra DB encrypts data. The BYOK feature enhances data security by allowing you to instead associate a Customer Key in Astra DB with a Customer Managed Encryption Key (CMEK) that you defined via a cloud provider, such as in Google Cloud KMS.
Adding a new Customer Key
Having completed the steps in Google Cloud KMS to set up a CMEK, and granted DataStax access to your Google Cloud Storage (GCS) buckets, follow these steps in Astra Portal to add a new Customer Key:
-
From your left navigation, click Settings, and navigate to Security Settings.
-
On Security Settings, if your account includes the necessary permissions and roles, notice Key Encryption, a Premium feature.
-
On Key Encryption, click Add Keys.
-
On Add Customer Key:
-
Choose Google Cloud Platform as the provider.
-
Choose the same region used by your Google Cloud KMS Customer Managed Encryption Key and the Astra DB database that will have its data encrypted. Example:
-
For the Key ID value, paste in the resource name of your Customer Managed Encryption Key, which you copied in Google Cloud KMS.
-
Click Add Key.
-
-
The Security Settings page displays the one or more Customer Key(s) defined in your organization. Example:
In the Status column:
-
In-Use means a database in your organization is actively using the Customer Key.
-
Available status indicates a Customer Key has been added for your organization, but no Astra DB database with that Provider and Region combination is using the key.
Customer Key status indicators
In addition to the status on Key Encryption in Security Settings (illustrated above), look for Customer Key status indicators on the Dashboard’s Overview tab.
Here’s an example with the key icon for the current database, indicating a Customer Key is used to protect its data:
A Region Details example with a Customer Key status indicator: