Auto-generate embeddings with vectorize
For Serverless (Vector) databases, you can bring your own embeddings or use Astra DB vectorize to automatically generate embeddings through an embedding provider integration:
| Bring your own embeddings |
Generate your own embeddings on the client side, and then import them when you load data. |
| Auto-generate embeddings with vectorize |
Configure an embedding provider integration, and then use vectorize to automatically generate embeddings from text for any operation that requires a vector. Enabling a vectorize integration also enables the Unstructured data loader integration. Vectorize doesn’t support embeddings for images. If you need embeddings for image data, you must bring your own embeddings. |
Learn more about embedding providers
Embedding providers are services that help you generate embeddings for your data to perform vector search queries.
The provider handles infrastructure, model maintenance, and other tasks necessary to generate embeddings from embedding models.
Providers may use one or more models to generate embeddings. When choosing an embedding provider, consider factors like the available embedding models, vector dimensions, supported data types, quality, accuracy, scalability, and so on.
Supported embedding providers
Vectorize generates embeddings through integrations with supported embedding providers:
| External providers |
Integrate your embedding provider account with Astra DB. |
| Astra-hosted providers |
Use a DataStax-managed embedding provider hosted within Astra DB. Databases in supported regions can automatically use these integrations for new collections. |
Astra DB Serverless offers the following embedding provider integrations:
| Embedding provider | External | Astra-hosted | Docs | |
|---|---|---|---|---|
|
Azure OpenAI |
|||
|
Hugging Face - Dedicated |
|||
|
Hugging Face - Serverless |
|||
|
Jina AI |
|||
|
Mistral AI |
|||
|
NVIDIA |
|||
|
OpenAI |
|||
|
Upstage |
|||
|
Voyage AI |
|||
External embedding provider integrations
To use an external embedding provider with Astra DB vectorize, you must add the embedding provider integration to your Astra DB organization, and then you can select your embedding provider when you create collections.
All providers follow the same general integration process. However, each provider has specific configuration options, such as models, dimensions, credentials, and other parameters. For complete instructions, see the documentation for your embedding provider.
|
Your external embedding provider integration uses your embedding provider account to generate embeddings. You can incur billed charges for this use according to your agreement with your provider. |
Embedding provider authentication
Astra DB needs authorization to request embeddings from your embedding provider. To do this, you provide credentials that authenticate Astra DB to your embedding provider account:
-
Add the external embedding provider integration to your Astra DB organization.
-
Add embedding provider credentials, such as API keys or access tokens.
-
Add databases to the credentials' scoped databases. This makes the integration available to new collections in the scoped databases.
-
Create a collection in a scoped database, select the embedding provider integration, and then select a credential from the pool of credentials that are available to the database.
If you can’t choose your integration or credential when you create a collection, see Potential scope delay.
-
Load data into the collection to auto-generate embeddings with vectorize. Astra DB vectorize uses the selected credential to request embeddings from your embedding provider.
For complete instructions, see the documentation for your embedding provider.
After you configure an embedding provider integration, you can manage the integration’s credentials and scoped databases in the Astra Portal. Go to Integrations, and then select the embedding provider integration that you want to manage. You can add or remove credentials and change the scoped databases for each credential.
|
When you remove a credential from Astra DB, the credential is not removed from your embedding provider account. Make sure to delete the credential from your embedding provider account if you no longer need it. Removing a credential immediately disables vectorize embedding generation in all collections that use the removed credential. |
For greater access control, you can add multiple credentials, and each credential can have different scoped databases. You can also add the same database to multiple credential scopes. However, when you create a collection, you choose only one integration and one credential, regardless of the number of integrations or credentials that are available to the database.
Scoped databases
Embedding provider integrations aren’t inherently available to all databases. You decide the databases and collections that can use each integration.
To make an external embedding provider integration available to a database, you must link the database to an embedding provider credential. When you do this, you add the database to the credential’s scoped databases. When you create a collection in a scoped database, you can choose from the integrations and credentials that are available to the database.
You can add a new database to a credential’s scope as soon as you create the database, even while the database is initializing. Once the database becomes active, you can create collections that use the external embedding provider integration.
Potential scope delay
Usually, when you add a database to a credential’s scope, the integration is almost immediately available to new collections that you create in the database. Rarely, Astra DB can take a few minutes to propagate a scope change. You might encounter an error or delay when you try to create a collection in the following scenarios:
-
When you add extensive multi-region databases to a credential’s scope, it can take time to propagate the change to all regions.
-
When you add a new embedding provider integration while creating a collection in the Astra Portal, it can take time to activate the integration. This includes adding the database to the new credential’s scope.
If you can’t select your integration when you create a collection, make sure the database is scoped to at least one of the provider’s credentials in Astra DB. If the database is in scope and you recently added the integration or credential, wait a few minutes and try again.
Change providers or credentials
When you create a vector-enabled collection, you select the collection’s embedding generation method, which can include an embedding provider integration and credential. These selections are permanent for the life of the collection.
To change a collection’s embedding provider, you must create a new collection that uses that integration.
To rotate credentials without changing embedding providers, you must remove the credential, and then recreate it with the same name and scoped databases:
-
In your embedding provider account, delete the old credential and create a new credential.
Removing a credential from either your embedding provider or Astra DB immediately disables
$vectorizeembedding generation for any collection that uses that credential. Vectorize remains unavailable until you add the new credential to the embedding provider integration. -
In the Astra Portal navigation menu, click Integrations, and then select the relevant embedding provider integration.
-
In the API keys section, locate the credential that you want to remove. Make a note of the credential’s name and scoped databases. When you recreate the credential, it must have the exact same name and scope.
-
Click More, and then select Remove API key. In the confirmation dialog, enter the API key name, and then click Remove key.
-
Click Add API key to add a new credential with the same API key name as the removed credential.
If the name doesn’t match, any collections that used the removed credential can’t detect the replacement.
-
Add all relevant databases to the new credential’s scoped databases.
At minimum, you must add all databases that used the removed credential so that the collections in those databases can detect the replacement. To ensure that you don’t miss any databases, DataStax recommends adding all of the databases that were in the removed credential’s scope.
Troubleshoot vectorize
When working with vectorize, including the $vectorize reserved field in the Data API, errors can occur from two sources:
- Astra DB
-
There is an issue within Astra DB, including the Astra DB platform, the Data API server, Data API clients, or something else.
Some of the most common Astra DB vectorize errors are related to scoped databases. In your vectorize integration settings, make sure your database is in the scope of the credential that you want to use. Scoped database errors don’t apply to the NVIDIA Astra-hosted embedding provider integration.
When using the Data API, make sure you don’t use
$vectorand$vectorizein the same query. For more information, see the Data API references, such as Vector and vectorize,insertMany, and Sort operations. - The embedding provider
-
The embedding provider encountered an issue while processing the embedding generation request. Astra DB passes these errors to you through the Astra Portal or Data API with a qualifying statement such as
The embedding provider returned a HTTP client error.Possible embedding provider errors include rate limiting, billing or account funding issues, chunk or token size limits, and so on. For more information about these errors, see the embedding provider’s documentation, including the documentation for your chosen model.
Carefully read all error messages to determine the source and possible cause for the issue.
