Integrate Hugging Face Dedicated as an embedding provider
Integrate Hugging Face Dedicated as an external embedding provider for Astra DB vectorize to leverage Hugging Face Dedicated’s embeddings API within Astra DB Serverless.
Prerequisites
To configure the Hugging Face Dedicated embedding provider integration, you need the following:
-
An active Astra account with the Organization Administrator role.
-
A Serverless (Vector) database.
If this is your first time using Astra DB, follow the Quickstart to create a database and connect to it with an API client.
-
A paid Hugging Face Dedicated account.
Create the Hugging Face Dedicated user access token
Sign in to your Hugging Face Dedicated account and create a new user access token with unrestricted access to the API. Make sure to copy the user access token to a secure location.
Don’t modify or delete the user access token in your Hugging Face Dedicated account after you’ve added it to Astra DB. This breaks the integration. For more information, see Embedding provider authentication. |
Add the Hugging Face Dedicated integration to your organization
Use the Astra Portal to add the Hugging Face Dedicated embedding provider integration to your Astra DB organization:
-
In the Astra Portal navigation menu, click Integrations.
-
In the All Integrations section, select Hugging Face Dedicated Embedding provider.
-
Click Add integration.
-
In the Add Integration dialog, do the following:
-
Enter a unique User access token name.
You can’t change user access token names. Make sure the name is meaningful and that it helps you identify your Hugging Face Dedicated user access token in Astra DB.
User access token names must start and end with a letter or number, and they can’t exceed 50 characters. Valid characters include letters, numbers, underscores (
_
), and hyphens (-
). -
Enter your Hugging Face Dedicated user access token.
-
In the Add databases to scope section, select a Serverless (Vector) database that you want to use the Hugging Face Dedicated user access token.
When you create a collection in a scoped database, you can choose any of the user access tokens that are available to the database. Astra DB uses the user access token to request embeddings from your embedding provider when you load data into the collection.
You can add up to 10 databases at once, and you can add more databases later.
For greater access control, you can add multiple user access tokens, and each user access token can have different scoped databases. Additionally, you can add the same database to multiple user access token scopes.
For example, you can have a few broadly-scoped user access tokens or many narrowly-scoped user access tokens.
For more information, see Embedding provider authentication, Scoped databases, and Manage scoped databases.
-
-
Click Add Integration.
The Hugging Face Dedicated integration switches to
Active, and your user access token and its scoped databases appear in the API keys section. If you want to add more user access tokens for this integration, click Add API key.
When you create collections in the scoped databases, you can select the Hugging Face Dedicated integration, and then use it to generate embeddings.
Add the Hugging Face Dedicated integration to a new collection
Before you can use the Hugging Face Dedicated integration to generate embeddings, you must add the integration to a new collection.
You can’t change a collection’s embedding provider or embedding generation method after you create it. To use a different embedding provider, you must create a new collection with a different embedding provider integration. |
-
Astra Portal
-
Python
-
TypeScript
-
Java
-
curl
-
In the Astra Portal, go to Databases, and then select your Serverless (Vector) database.
-
Click Data Explorer.
-
In the Namespace field, select the namespace where you want to create the collection, or use the default namespace, which is named
default_keyspace
. -
Click Create Collection.
-
In the Create collection dialog, enter a name for the collection. Collection names can contain no more than 50 characters, including letters, numbers, and underscores.
-
Turn on Vector-enabled collection.
-
Under Embedding generation method, select the Hugging Face Dedicated embedding provider integration.
If the integration isn’t listed, follow the steps in Add the Hugging Face Dedicated integration to your organization and Manage scoped databases to make sure the integration is active and that your database is scoped to at least one user access token.
-
Complete the following fields:
-
User access token: The user access token that you want the collection to use to access your embedding provider and generate embeddings. This field is only active if the database is scoped to multiple Hugging Face Dedicated user access tokens.
-
Endpoint name: The programmatically-generated name of your Hugging Face Dedicated endpoint. This is the first part of the endpoint URL. For example, if your endpoint URL is
https://mtp1x7muf6qyn3yh.us-east-1.aws.endpoints.huggingface.cloud
, the endpoint name ismtp1x7muf6qyn3yh
. -
Region name: The cloud provider region your Hugging Face Dedicated endpoint is deployed to. For example,
us-east-1
. -
Cloud name: The cloud provider your Hugging Face Dedicated endpoint is deployed to. For example,
aws
. -
Embedding model: The model that you want to use to generate embeddings. The available models are:
endpoint-defined-model
. For Hugging Face Dedicated, the integration uses the model that you defined in your dedicated endpoint configuration. -
Dimensions: The number of dimensions that you want the generated vectors to have. Most models automatically populate the Dimensions. You can edit this field if the model supports a range of dimensions or the embedding provider integration uses an endpoint-defined model. Your chosen embedding model must support the specified number of dimensions.
-
Similarity metric: The method you want to use to calculate vector similarity scores. The available metrics are Cosine, Dot Product, and Euclidean.
-
-
Click Create collection.
Use the Python client to create a collection that uses the Hugging Face Dedicated integration.
Initialize the client
If you haven’t done so already, initialize the client before creating a collection:
import os
from astrapy import DataAPIClient
from astrapy.constants import VectorMetric
from astrapy.info import CollectionVectorServiceOptions
# Initialize the client and get a "Database" object
client = DataAPIClient(os.environ["ASTRA_DB_APPLICATION_TOKEN"])
database = client.get_database(os.environ["ASTRA_DB_API_ENDPOINT"])
print(f"* Database: {database.info().name}\n")
Create a collection integrated with Hugging Face Dedicated:
collection = database.create_collection(
"COLLECTION_NAME",
metric=VectorMetric.COSINE,
dimension=MODEL_DIMENSIONS,
service=CollectionVectorServiceOptions(
provider="huggingfaceDedicated",
model_name="endpoint-defined-model",
authentication={
"providerKey": "API_KEY_NAME",
},
parameters={
"endpointName": "ENDPOINT_NAME",
"regionName": "REGION_NAME",
"cloudName": "CLOUD_NAME",
},
),
)
print(f"* Collection: {collection.full_name}\n")
Replace the following:
-
COLLECTION_NAME
: The name for your collection. -
API_KEY_NAME
: The name of the Hugging Face Dedicated user access token that you want to use for your collection. Must be the name of an existing Hugging Face Dedicated user access token in the Astra Portal. -
MODEL_NAME
: The desired model to use to generate embeddings. For Hugging Face Dedicated, the supported models areendpoint-defined-model
. -
MODEL_DIMENSIONS
: The number of dimensions that you want the generated vectors to have. Your embedding model must support the specified number of dimensions.If your model has a default dimension value, you can omit
dimension
.You can use the Data API to find supported embedding providers and their configuration parameters, including dimensions ranges and default dimensions.
-
ENDPOINT_NAME
: The programmatically-generated name of your Hugging Face Dedicated endpoint. This is the first part of the endpoint URL. For example, if your endpoint URL ishttps://mtp1x7muf6qyn3yh.us-east-1.aws.endpoints.huggingface.cloud
, the endpoint name ismtp1x7muf6qyn3yh
. -
REGION_NAME
: The cloud provider region your Hugging Face Dedicated endpoint is deployed to. For example,us-east-1
. -
CLOUD_NAME
: The cloud provider your Hugging Face Dedicated endpoint is deployed to. For example,aws
.
The model name must be set to |
Use the TypeScript client to create a collection that uses the Hugging Face Dedicated integration.
Initialize the client
If you haven’t done so already, initialize the client before creating a collection:
import { DataAPIClient, VectorDoc, UUID } from '@datastax/astra-db-ts';
const { ASTRA_DB_APPLICATION_TOKEN, ASTRA_DB_API_ENDPOINT } = process.env;
// Initialize the client and get a 'Db' object
const client = new DataAPIClient(ASTRA_DB_APPLICATION_TOKEN);
const db = client.db(ASTRA_DB_API_ENDPOINT);
console.log(`* Connected to DB ${db.id}`);
Create a collection integrated with Hugging Face Dedicated:
(async function () {
const collection = await db.createCollection('COLLECTION_NAME', {
vector: {
dimension: MODEL_DIMENSIONS,
service: {
provider: 'huggingfaceDedicated',
modelName: 'endpoint-defined-model',
authentication: {
providerKey: 'API_KEY_NAME',
},
parameters: {
endpointName: 'ENDPOINT_NAME',
regionName: 'REGION_NAME',
cloudName: 'CLOUD_NAME',
},
},
},
});
console.log(`* Created collection ${collection.keyspace}.${collection.collectionName}`);
Replace the following:
-
COLLECTION_NAME
: The name for your collection. -
API_KEY_NAME
: The name of the Hugging Face Dedicated user access token that you want to use for your collection. Must be the name of an existing Hugging Face Dedicated user access token in the Astra Portal. -
MODEL_NAME
: The desired model to use to generate embeddings. For Hugging Face Dedicated, the supported models areendpoint-defined-model
. -
MODEL_DIMENSIONS
: The number of dimensions that you want the generated vectors to have. Your embedding model must support the specified number of dimensions.If your model has a default dimension value, you can omit
dimension
.You can use the Data API to find supported embedding providers and their configuration parameters, including dimensions ranges and default dimensions.
-
ENDPOINT_NAME
: The programmatically-generated name of your Hugging Face Dedicated endpoint. This is the first part of the endpoint URL. For example, if your endpoint URL ishttps://mtp1x7muf6qyn3yh.us-east-1.aws.endpoints.huggingface.cloud
, the endpoint name ismtp1x7muf6qyn3yh
. -
REGION_NAME
: The cloud provider region your Hugging Face Dedicated endpoint is deployed to. For example,us-east-1
. -
CLOUD_NAME
: The cloud provider your Hugging Face Dedicated endpoint is deployed to. For example,aws
.
The model name must be set to |
Use the Java client to create a collection that uses the Hugging Face Dedicated integration.
Initialize the client
If you haven’t done so already, initialize the client before creating a collection:
import com.datastax.astra.client.Collection;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.Database;
import com.datastax.astra.client.model.CollectionOptions;
import com.datastax.astra.client.model.Document;
import com.datastax.astra.client.model.FindIterable;
import com.datastax.astra.client.model.FindOptions;
import com.datastax.astra.client.model.SimilarityMetric;
import static com.datastax.astra.client.model.SimilarityMetric.COSINE;
public class Quickstart {
public static void main(String[] args) {
// Loading Arguments
String astraToken = System.getenv("ASTRA_DB_APPLICATION_TOKEN");
String astraApiEndpoint = System.getenv("ASTRA_DB_API_ENDPOINT");
// Initialize the client
DataAPIClient client = new DataAPIClient(astraToken);
System.out.println("Connected to AstraDB");
Database db = client.getDatabase(astraApiEndpoint);
System.out.println("Connected to Database.");
Create a collection integrated with Hugging Face Dedicated:
Map<String, Object > params = new HashMap<>();
params.put("endpointName", "ENDPOINT_NAME");
params.put("regionName", "REGION_NAME");
params.put("cloudName", "CLOUD_NAME");
CollectionOptions.CollectionOptionsBuilder builder = CollectionOptions
.builder()
.vectorSimilarity(SimilarityMetric.COSINE)
.vectorDimension(MODEL_DIMENSIONS)
.vectorize("huggingfaceDedicated", "endpoint-defined-model", "API_KEY_NAME", params);
Collection<Document> collection = db
.createCollection("COLLECTION_NAME", builder.build());
Replace the following:
-
COLLECTION_NAME
: The name for your collection. -
API_KEY_NAME
: The name of the Hugging Face Dedicated user access token that you want to use for your collection. Must be the name of an existing Hugging Face Dedicated user access token in the Astra Portal. -
MODEL_NAME
: The desired model to use to generate embeddings. For Hugging Face Dedicated, the supported models areendpoint-defined-model
. -
MODEL_DIMENSIONS
: The number of dimensions that you want the generated vectors to have. Your embedding model must support the specified number of dimensions.If your model has a default dimension value, you can omit
dimension
.You can use the Data API to find supported embedding providers and their configuration parameters, including dimensions ranges and default dimensions.
-
ENDPOINT_NAME
: The programmatically-generated name of your Hugging Face Dedicated endpoint. This is the first part of the endpoint URL. For example, if your endpoint URL ishttps://mtp1x7muf6qyn3yh.us-east-1.aws.endpoints.huggingface.cloud
, the endpoint name ismtp1x7muf6qyn3yh
. -
REGION_NAME
: The cloud provider region your Hugging Face Dedicated endpoint is deployed to. For example,us-east-1
. -
CLOUD_NAME
: The cloud provider your Hugging Face Dedicated endpoint is deployed to. For example,aws
.
The model name must be set to |
Use the Data API to create a collection that uses the Hugging Face Dedicated integration:
curl -sS --location -X POST "$ASTRA_DB_API_ENDPOINT/api/json/v1/default_keyspace" \
--header "Token: $ASTRA_DB_APPLICATION_TOKEN" \
--header "Content-Type: application/json" \
--data '{
"createCollection": {
"name": "COLLECTION_NAME",
"options": {
"vector": {
"metric": "cosine",
"dimension": MODEL_DIMENSIONS,
"service": {
"provider": "huggingfaceDedicated",
"modelName": "endpoint-defined-model",
"authentication": {
"providerKey": "API_KEY_NAME"
},
"parameters": {
"endpointName": "ENDPOINT_NAME",
"regionName": "REGION_NAME",
"cloudName": "CLOUD_NAME"
}
}
}
}
}
}' | jq
Replace the following:
-
COLLECTION_NAME
: The name for your collection. -
API_KEY_NAME
: The name of the Hugging Face Dedicated user access token that you want to use for your collection. Must be the name of an existing Hugging Face Dedicated user access token in the Astra Portal. -
MODEL_NAME
: The desired model to use to generate embeddings. For Hugging Face Dedicated, the supported models areendpoint-defined-model
. -
MODEL_DIMENSIONS
: The number of dimensions that you want the generated vectors to have. Your embedding model must support the specified number of dimensions.If your model has a default dimension value, you can omit
dimension
.You can use the Data API to find supported embedding providers and their configuration parameters, including dimensions ranges and default dimensions.
-
ENDPOINT_NAME
: The programmatically-generated name of your Hugging Face Dedicated endpoint. This is the first part of the endpoint URL. For example, if your endpoint URL ishttps://mtp1x7muf6qyn3yh.us-east-1.aws.endpoints.huggingface.cloud
, the endpoint name ismtp1x7muf6qyn3yh
. -
REGION_NAME
: The cloud provider region your Hugging Face Dedicated endpoint is deployed to. For example,us-east-1
. -
CLOUD_NAME
: The cloud provider your Hugging Face Dedicated endpoint is deployed to. For example,aws
.
The model name must be set to |
If you get a Serverless (Vector) databases created after June 24, 2024 can have up to 10 collections. Databases created before this date can have up to 5 collections. The collection limit is based on Storage Attached Indexing (SAI). |
After you create a collection, load data into the collection.
Load and search data with vectorize
-
Load vector data into your vectorize-integrated collection.
When you load structured JSON or CSV data, the Vector Field specifies field to use to generate embeddings with
$vectorize
. -
After loading data, you can perform a similarity search using text, rather than a vector.
Manage scoped databases
For each user access token, you select the databases that can use that user access token. These are referred to as scoped databases.
To change the scoped databases for an existing Hugging Face Dedicated user access token, do the following:
-
In the Astra Portal navigation menu, click Integrations, and then select Hugging Face Dedicated.
-
In the API keys section, expand each user access token to show the list of scoped databases.
-
Add or remove databases from each user access token’s scope, as needed:
-
To remove a database from the user access token’s scope, click
Delete, enter the Database name, and then click Remove scope. -
To add a database to the user access token’s scope, click
More, select Add database, select the Serverless (Vector) database that you want to add to the scope, and then click Add database.
-
Remove Hugging Face Dedicated user access tokens
Removing user access tokens immediately disables Removing user access tokens from Astra DB Serverless does not delete them from your Hugging Face Dedicated account. |
To remove user access tokens, do the following:
-
In the Astra Portal navigation menu, click Integrations, and then select Hugging Face Dedicated Embedding provider.
-
In the API keys section, locate the user access token that you want to remove, click
More, and then select Remove API key. -
In the confirmation dialog, enter the API key name, and then click Remove key.
-
In your Hugging Face Dedicated account, delete the user access token if you don’t plan to reuse it.
-
If you no longer want to use this embedding provider or you are not rotating the user access token, then you must recreate any collections that used the removed user access token to generate embeddings. For more information, see Change providers or credentials.
Rotate Hugging Face Dedicated user access tokens
To rotate user access tokens, you must remove the user access token, and then recreate it with the same name and scoped databases.
Removing the user access token immediately disables $vectorize
embedding generation for any collections that used that user access token.
Vectorize remains unavailable until you add the new user access token to the Hugging Face Dedicated integration.
For more information, see Change providers or credentials.
To rotate user access tokens, do the following:
-
In your Hugging Face Dedicated, create a new user access token.
-
Remove the user access token that you want to rotate. Make a note of the user access token’s name and scoped databases. When you recreate the user access token, it must have the exact same name and scope.
-
In the Astra Portal navigation menu, click Integrations, and then select Hugging Face Dedicated Embedding provider.
-
In the API keys section, add a new user access token with the same name as the removed user access token.
If the name doesn’t match, any collections that used the removed user access token can’t detect the replacement user access token.
-
Add all relevant databases to the new user access token’s scoped databases.
At minimum, you must add all databases that used the removed user access token so that the collections in those databases can detect the replacement user access token. To ensure that you don’t miss any databases, DataStax recommends adding all of the databases that were in the removed user access token’s scope.
Remove the Hugging Face Dedicated integration from your organization
To remove the Hugging Face Dedicated embedding provider integration from your Astra organization remove all existing Hugging Face Dedicated user access tokens, and then recreate any collections that used the integration to generate embeddings.
Troubleshoot vectorize
When working with vectorize, including the $vectorize
reserved field in the Data API, errors can occur from two sources:
- Astra DB
-
There is an issue within Astra DB, including the Astra DB platform, the Data API server, Data API clients, or something else.
Some of the most common Astra DB vectorize errors are related to scoped databases. In your vectorize integration settings, make sure your database is in the scope of the credential that you want to use. Scoped database errors don’t apply to the NVIDIA Astra-hosted embedding provider integration.
When using the Data API, make sure you don’t use
$vector
and$vectorize
in the same query. For more information, see the Data API references, such as Vector and vectorize,insertMany
, and Sort operations. - The embedding provider
-
The embedding provider encountered an issue while processing the embedding generation request. Astra DB passes these errors to you through the Astra Portal or Data API with a qualifying statement such as
The embedding provider returned a HTTP client error
.Possible embedding provider errors include rate limiting, billing or account funding issues, chunk or token size limits, and so on. For more information about these errors, see the embedding provider’s documentation, including the documentation for your chosen model.
Carefully read all error messages to determine the source and possible cause for the issue.