Manage embedding provider integrations for vectorize
There are two ways to load embeddings into Serverless (vector) databases:
-
Bring your own embeddings: Generate your own embeddings on the client side, and then import them when you insert data or perform a vector search.
-
Auto-generate embeddings with vectorize: Configure a supported embedding provider integration, and then use vectorize to automatically generate embeddings when you insert data or perform a vector search. Enabling a vectorize integration also enables the Unstructured data loader integration.
For supported embedding providers, see Astra-hosted embedding provider integrations and External embedding provider integrations. To use a provider, model, or model setting that isn’t available through Astra vectorize, you must bring your own embeddings.
Vectorize integrations generate embeddings for text data only. If you need embeddings for images or other media, you must bring your own embeddings.
How does vectorize work
The following steps summarize the process to configure and use a vectorize embedding provider integration. For complete instructions and examples, see the documentation for your preferred Astra-hosted embedding provider or external embedding provider integration.
-
Enable the embedding provider integration in your Astra organization.
Astra-hosted embedding provider integrations are enabled by default, and Astra manages authentication to the service. These integrations are automatically available to qualifying databases.
External embedding provider integrations connect to your own embedding provider account. You must provide credentials, like API keys and user access tokens, and then authorize your databases to use those credentials. This makes the integration available to collections and tables in the scoped databases. For more information, see Manage scoped databases and External embedding provider authentication.
-
Attach the integration to a collection or table:
-
Collections: You can select a vectorize integration when you create a new collection only. You cannot add a vectorize integration to an existing collection.
-
Tables: You can add a vectorize integration to a new or existing table with the Data API. Then, create a vector index to enable vector search.
-
-
Insert data into the collection or table to auto-generate embeddings with vectorize. The vectorize integration calls the embedding provider’s API to generate embeddings for specified data.
Astra determines which data to embed based on reserved field names or column types. For collections, vector embeddings are automatically generated when you insert a document with a
$vectorizefield. For tables, vector embeddings are automatically generated when you insert a row with a string value into the integrated vector column. For more information, see Ways to insert data in Astra DB Serverless. -
Perform a vector search on the embedded data with a natural language query. Astra uses the vectorize integration to generate an embedding from your query string, and then runs the vector search. For more information, see Find data with vector search.
Astra-hosted embedding provider integrations
Vectorize generates embeddings through supported embedding provider integrations.
DataStax-managed embedding provider integrations are hosted within Astra. These integrations don’t require your own embedding provider account or credentials because they are managed by Astra. However, there are restrictions on the available regions and configuration options.
Databases in supported regions can configure collections and tables to automatically use these integrations:
| Embedding provider | Documentation |
|---|---|
NVIDIA |
External embedding provider integrations
|
An external embedding provider integration uses your embedding provider account to generate embeddings. You can incur billed charges for this use according to your agreement with your provider. |
To use an external embedding provider with Astra vectorize, you must attach your embedding provider account to your Astra organization by enabling the embedding provider integration in your Astra organization. Then, you can attach the embedding provider integration to a collection or table.
All providers follow the same general integration process. However, each provider has specific configuration options, such as models, dimensions, credentials, and other parameters. For complete instructions, see the documentation for your embedding provider:
| Embedding provider | Documentation |
|---|---|
Azure OpenAI |
|
Hugging Face - Dedicated |
|
Hugging Face - Serverless |
|
Jina AI |
|
Mistral AI |
|
OpenAI |
|
Upstage |
|
Voyage AI |
Manage scoped databases
By default, embedding provider integrations aren’t available to all databases:
-
Astra-hosted embedding provider integrations: Automatically available to qualifying databases only. You cannot activate these integrations for ineligible databases. For more information, see the documentation for your preferred integration.
-
External embedding provider integrations: You decide which databases, collections, and tables can use each integration. To do this, you add embedding provider credentials and specify the databases that can use each credential. Then, you can add the integration to collections and tables in the specified databases.
A credential’s authorized databases are known as scoped databases. Databases within a credential’s scope can use a credential to call the associated embedding provider’s API.
|
External embedding provider integrations aren’t retroactively applied to collections and tables in scoped databases. You must intentionally configure collections and tables use an integration after adding credentials and scoped databases. For information about adding integrations to collections and tables, see the documentation for your preferred provider. Additionally, new databases aren’t automatically added to any credential scopes. When you create a database, you must add it to the relevant credential scopes if you want to use vectorize integrations with that database. You can add a new database to a credential’s scope as soon as you create the database, even while the database is initializing. |
To change the scoped databases for an external embedding provider integration, do the following:
-
In the Astra Portal header, click Settings.
-
In the Settings navigation menu, make sure the enterprise/organization filter is set to the organization that you want to manage.
If the organization belongs to an enterprise, you must filter on the enterprise, and then click the organization name in the Organizations list.
-
In the Settings navigation menu, click Integrations.
-
Click the integration that you want to manage.
-
In the API keys section, expand a credential to view the credential’s scoped databases.
Each credential has its own list of scoped databases.
-
Add or remove databases from each credential’s scope, as needed:
-
To remove a database from a credential’s scope, click delete Delete, enter the Database name, and then click Remove scope.
-
To add a database to a credential’s scope, click More, select Add database, select the Serverless (vector) database that you want to add, and then click Add database.
You can add up to 10 databases at once. To add more than 10 databases, edit the credential’s scoped databases repeatedly until you have added all relevant databases.
-
Potential scope delay
Usually, when you add a database to a credential’s scope, the integration is available to the database almost immediately. Rarely, Astra can take a few minutes to propagate a scope change. Typically, this delay occurs when you add a new embedding provider integration while creating a collection in the Astra Portal because it can take time to activate the integration in your organization and the scoped databases.
If a particular integration or credential isn’t available to a collection or table, make sure the database is scoped to at least one of the provider’s credentials in Astra. If the database is in scope and you recently added the integration or credential, wait a few minutes and try again.
External embedding provider authentication
For Astra-hosted embedding provider integrations, Astra manages authentication, credentials, and scoped databases automatically. However, these integrations are limited to qualifying databases only. For more information, see the documentation for your preferred integration.
For external embedding provider integrations, you must provide credentials that authenticate Astra to your embedding provider account. For initial setup and provider-specific authentication requirements, see the documentation for your preferred integration.
|
Editing and removing credentials and scoped databases can impact an integration’s availability. |
Use multiple credentials
For greater access control, you can add multiple credentials to each embedding provider integration, and each credential can have different scoped databases. You can also add the same database to multiple credential scopes. For example, you can have a few broadly-scoped credentials or many narrowly-scoped credentials.
However, regardless of the number of integrations or credentials that are available to a database, there is a one-to-one relationship between embedding provider integrations and credentials when attached to a collection or a vector column in a table:
-
Collections: When you create a collection, you select only one integration and one credential for the collection. These selections are permanent for the life of the collection.
-
Tables: For each vector column in a table, you can attach one integration and one credential. If the table has multiple vector columns, you can select a different integration and credential for each column. Additionally, you can add and remove embedding provider integrations from vector columns at any time.
Rotate credentials
To rotate embedding provider credentials, you must remove the existing credential in Astra, and then add a new credential with the same name and scoped databases:
-
In your embedding provider account, create a new credential.
-
In the Astra Portal header, click Settings.
-
In the Settings navigation menu, make sure the enterprise/organization filter is set to the organization that you want to manage.
If the organization belongs to an enterprise, you must filter on the enterprise, and then click the organization name in the Organizations list.
-
In the Settings navigation menu, click Integrations.
-
Click the integration that you want to manage.
-
In the API keys section, find the credential that you want to rotate. Make a note of the credential’s name and scoped databases.
When you re-create the credential, it must have the exact same name and scope.
If you want to change the credential’s name or scoped databases, follow the steps to Change the embedding provider configuration for a collection or table.
-
Click More, and then select Remove API key. In the confirmation dialog, enter the API key name, and then click Remove key.
Removing a credential immediately disables vectorize embedding generation for all collections and tables that used the removed credential. Vectorize remains unavailable until you add the replacement credential in the integration settings.
-
Click Add API key to add a new credential with the same name as the removed credential.
If the name doesn’t match, any collections and tables that used the removed credential cannot detect the replacement credential.
-
Add all relevant databases to the new credential’s scoped databases.
At minimum, you must add all databases that used the removed credential so that the collections and tables in those databases can detect the replacement credential. To ensure that you don’t miss any databases, DataStax recommends adding all of the databases that were in the removed credential’s scope.
You can add up to 10 databases at once. To add more than 10 databases, edit the credential’s scoped databases repeatedly until you have added all relevant databases.
-
Remove the unused credential from your embedding provider account if it is no longer needed.
Remove credentials
Use these steps to completely remove a credential from an external embedding provider integration in Astra. To replace a credential, see Rotate credentials.
|
Removing a credential immediately disables vectorize embedding generation for all collections and tables that used the removed credential. Before you remove a credential, make sure that no collections or tables are actively using the credential. Removing credentials from Astra doesn’t remove them from your embedding provider account. It is your responsibility to remove unused credentials from your embedding provider account. |
-
In the Astra Portal header, click Settings.
-
In the Settings navigation menu, make sure the enterprise/organization filter is set to the organization that you want to manage.
If the organization belongs to an enterprise, you must filter on the enterprise, and then click the organization name in the Organizations list.
-
In the Settings navigation menu, click Integrations.
-
Click the integration that you want to manage.
-
In the API keys section, find the credential that you want to remove, click More, and then select Remove API key.
-
In the confirmation dialog, enter the API key name, and then click Remove key.
-
In your embedding provider account, delete the removed credential if it is no longer needed.
-
If any collections or tables used the removed credential, you must remove or replace the integration on those collections and tables. For instructions, see Change the embedding provider configuration for a collection or table.
Change the embedding provider configuration for a collection or table
Use these guidelines to change the embedding provider configuration for a collection or table. This includes switching the vectorize integration, replacing vectorize with bring-your-own embeddings, or changing the model, dimensions, credential name, or other core settings for an existing vectorize integration.
-
Collections: When you create a vector-enabled collection, you select the collection’s embedding generation method, which can include a vectorize embedding provider integration, model, dimensions, and credential.
These selections are permanent for the life of the collection.
To change a collection’s embedding provider configuration, you must create a new collection with the new configuration.
To preserve your data, you can export the original collection’s data, and then load it into the new collection. If you plan to use a different embedding provider, model, or dimensions, make sure you remove the
$vectordata before reuploading the data. For examples, see Migrate to a new embedding model for a collection. -
Tables: To change the embedding provider configuration for a vector column in a table, you can alter the table.
The recommended approach is to create a new vector column with the new embedding provider configuration, generate or insert new embeddings, and then drop the original vector column if you no longer need it. For examples, see Migrate to a new embedding model for a table.
Alternatively, you can alter the existing vector column to use the new embedding provider configuration, but this approach can cause errors if the existing embeddings are incompatible with the new configuration.
Deactivate an integration
To remove an embedding provider integration from your Astra organization, do the following:
-
Remove the integration from any applicable collections and tables. For instructions, see Change the embedding provider configuration for a collection or table.
Troubleshoot vectorize integrations
When working with vectorize, including the $vectorize reserved field in the Data API, errors can arise from Astra or the embedding provider.
Astra errors can originate from the Astra platform, the Data API server, Data API clients, or another Astra component.
Embedding provider errors originate from the embedding provider service when Astra calls the provider’s API to generate embeddings.
Database not in scope
Some of the most common Astra DB vectorize errors are related to scoped databases. If an integration is unavailable or stops working, check the vectorize integration’s settings to verify that your database is in the scope of the credential that you want to use.
Scoped database errors cannot occur for Astra-hosted embedding provider integrations.
Embedding provider not available
There are several reasons why you cannot add a certain embedding provider to a collection or table:
-
For Astra-hosted embedding provider integrations, your database must be deployed to a supported region. Databases deployed to unsupported regions cannot access the integration.
-
When creating a collection in the Astra Portal, the listed external embedding provider integrations depend on the scoped databases. If you select Add embedding provider integration, you can configure a new integration for the first time only. After the initial configuration, you must manage the integration’s scoped databases through Settings.
-
If your preferred embedding provider isn’t listed on the Astra Integrations page, then there is no vectorize integration available for that provider. In this case, you can bring your own embeddings: Generate embeddings with your preferred provider outside of Astra, and then include the embeddings when you insert data. You can implement logic in your application’s code to facilitate embedding generation similar to vectorize, but you must generate the embeddings before writing data or sending a vector search query to Astra.
$vector and $vectorize are mutually exclusive
When using the Data API with collections, make sure you don’t use $vector and $vectorize in the same query.
For more information, see the Data API reference for collections.
Vector search on tables
When using the Data API with tables, you can only run a vector search on one vector column at a time.
To generate an embedding from a string, the target vector column must have a defined embedding provider integration. Otherwise, you must provide an explicit query embedding for the vector search.
For more information, see the Data API tables reference, such as Sort clauses for tables.
Permissions required for vectorize
The required permissions depend on the operations you need to perform:
- Manage integration credentials and scoped databases
-
To manage vectorize embedding provider integrations, you need the Organization Administrator role or a custom role with the Write Integrations permission. This isn’t required for Astra-hosted embedding provider integrations because they are automatically available to qualifying databases.
- Create databases
-
If you need to create a database to access a Astra-hosted embedding provider integration, you need the Database Administrator role or a custom role with the Create DB permission. However, if you create a database where you want to use an external embedding provider integration, you also need the Write Integrations permission to add the new database to the integration’s scoped databases. If you don’t have this permission, then you must contact your Organization Administrator for assistance.
- Add integrations to collections and tables
-
To add vectorize embedding provider integrations to collections and tables, you need the Database Administrator role or a custom role with permissions to create collections, create tables, alter tables, and create indexes for tables.
- Insert data and perform vector searches
-
For these permissions, see Ways to insert data in Astra DB Serverless and Ways to find data in Astra DB Serverless.
Embedding provider returned an error
Errors can occur within the embedding provider service while processing an embedding generation request.
Astra DB passes these errors to you through the Astra Portal or Data API with a qualifying statement such as The embedding provider returned a HTTP client error.
Read the entire error message to determine the source and possible cause for the issue.
Common embedding provider errors include rate limiting, billing or account funding issues, and chunk or token size limits.
For example, if the token limit is 1000 character, but your $vectorize strings are 1500 characters, then the embedding provider might reject the request.
For more information about these errors and limits, see the embedding provider’s documentation, including the documentation for your chosen model.
