Integrate NVIDIA as an embedding provider

Integrate the Astra-hosted NVIDIA embedding provider to enable Astra DB vectorize.

Prerequisites

To use NVIDIA as an embedding provider, you need the following:

Add the NVIDIA integration to collections and tables

You can use the NVIDIA integration in collections and tables.

Add the NVIDIA integration to a new collection

Before you can use the NVIDIA integration to generate embeddings, you must add the integration to a new collection.

You can’t change a collection’s embedding provider or embedding generation method after you create it. To use a different embedding provider, you must create a new collection with a different embedding provider integration.

  • Astra Portal

  • Python

  • TypeScript

  • Java

  • curl

  1. In the Astra Portal, go to Databases, and then select your Serverless (Vector) database.

  2. Click Data Explorer.

  3. In the Keyspace field, select the keyspace where you want to create the collection or use default_keyspace.

  4. Click Create Collection.

  5. In the Create collection dialog, enter a name for the collection. Collection names can contain no more than 50 characters, including letters, numbers, and underscores.

  6. Turn on Vector-enabled collection.

  7. For Embedding generation method, select the NVIDIA embedding provider integration.

  8. Complete the following fields:

    • Embedding model: The model that you want to use to generate embeddings. If only one model is available, it is selected by default.

    • Dimensions: The number of dimensions that you want the generated vectors to have. Typically, the number of dimensions is automatically determined by the model you select.

    • Similarity metric: The method you want to use to calculate vector similarity scores. The available metrics are Cosine, Dot Product, and Euclidean.

  9. Click Create collection.

Use the Python client to create a collection that uses the NVIDIA integration.

Initialize the client

If you haven’t done so already, initialize the client before creating a collection:

import os
from astrapy import DataAPIClient
from astrapy.constants import VectorMetric
from astrapy.info import CollectionVectorServiceOptions

# Initialize the client and get a "Database" object
client = DataAPIClient(os.environ["ASTRA_DB_APPLICATION_TOKEN"])
database = client.get_database(os.environ["ASTRA_DB_API_ENDPOINT"])
print(f"* Database: {database.info().name}\n")

Create a collection integrated with NVIDIA:

collection = database.create_collection(
    "COLLECTION_NAME",
    metric=VectorMetric.SIMILARITY_METRIC,
    service=CollectionVectorServiceOptions(
        provider="nvidia",
        model_name="NV-Embed-QA",
    ),
)
print(f"* Collection: {collection.full_name}\n")

Use the TypeScript client to create a collection that uses the NVIDIA integration.

Initialize the client

If you haven’t done so already, initialize the client before creating a collection:

import { DataAPIClient, VectorDoc, UUID } from '@datastax/astra-db-ts';

const { ASTRA_DB_APPLICATION_TOKEN, ASTRA_DB_API_ENDPOINT } = process.env;

// Initialize the client and get a 'Db' object
const client = new DataAPIClient(ASTRA_DB_APPLICATION_TOKEN);
const db = client.db(ASTRA_DB_API_ENDPOINT);

console.log(`* Connected to DB ${db.id}`);

Create a collection integrated with NVIDIA:

(async function () {
  const collection = await db.createCollection('COLLECTION_NAME', {
    vector: {
      service: {
        provider: 'nvidia',
        modelName: 'NV-Embed-QA',
      },
    },
  });
  console.log(`* Created collection ${collection.keyspace}.${collection.collectionName}`);

Use the Java client to create a collection that uses the NVIDIA integration.

Initialize the client

If you haven’t done so already, initialize the client before creating a collection:

import com.datastax.astra.client.Collection;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.Database;
import com.datastax.astra.client.model.CollectionOptions;
import com.datastax.astra.client.model.Document;
import com.datastax.astra.client.model.FindIterable;
import com.datastax.astra.client.model.FindOptions;
import com.datastax.astra.client.model.SimilarityMetric;

// Replace SIMILARITY_METRIC with your desired similarity metric:
// COSINE, DOT_PRODUCT, EUCLIDEAN
import static com.datastax.astra.client.model.SimilarityMetric.SIMILARITY_METRIC;

public class Quickstart {

  public static void main(String[] args) {
    // Loading Arguments
    String astraToken = System.getenv("ASTRA_DB_APPLICATION_TOKEN");
    String astraApiEndpoint = System.getenv("ASTRA_DB_API_ENDPOINT");

    // Initialize the client
    DataAPIClient client = new DataAPIClient(astraToken);
    System.out.println("Connected to AstraDB");

    Database db = client.getDatabase(astraApiEndpoint);
    System.out.println("Connected to Database.");

Create a collection integrated with NVIDIA:

CollectionOptions.CollectionOptionsBuilder builder = CollectionOptions
       .builder()
       .vectorSimilarity(SimilarityMetric.SIMILARITY_METRIC)
       .vectorize("nvidia", "NV-Embed-QA");
Collection<Document> collection = db
       .createCollection("COLLECTION_NAME", builder.build());

Use the Data API to create a collection that uses the NVIDIA integration:

curl -sS -L -X POST "$ASTRA_DB_API_ENDPOINT/api/json/v1/default_keyspace" \
--header "Token: $ASTRA_DB_APPLICATION_TOKEN" \
--header "Content-Type: application/json" \
--data '{
  "createCollection": {
    "name": "COLLECTION_NAME",
    "options": {
      "vector": {
        "metric": "cosine",
        "service": {
          "provider": "nvidia",
          "modelName": "NV-Embed-QA"
        }
      }
    }
  }
}' | jq

If you get a Collection Limit Reached or TOO_MANY_INDEXES message, you must delete a collection before you can create a new one.

Serverless (Vector) databases created after June 24, 2024 can have approximately 10 collections. Databases created before this date can have approximately 5 collections. The collection limit is based on the number of indexes.

Add the NVIDIA integration to a table

You can use the Data API to add the NVIDIA integration to a table in multiple ways:

  • Add the integration to a vector column when you create a table.

  • Add the integration to an existing vector column in a table.

  • Add the integration when you add a vector column to an existing table.

If you are new to the Data API, see Get started with the Data API.

Add the integration to a new table

  • Python

  • TypeScript

  • Java

  • curl

Use the Python client to create a table with a column that is integrated with NVIDIA:

import os
from astrapy import DataAPIClient
from astrapy.constants import VectorMetric
from astrapy.info import (
    CreateTableDefinition,
    ColumnType,
    VectorServiceOptions,
    TableVectorColumnTypeDescriptor,
    TableScalarColumnTypeDescriptor,
    TablePrimaryKeyDescriptor,
    TableVectorIndexOptions
)

# Initialize the client and get a "Database" object
client = DataAPIClient(os.environ["ASTRA_DB_APPLICATION_TOKEN"])
database = client.get_database(os.environ["ASTRA_DB_API_ENDPOINT"])
print(f"* Database: {database.info().name}\n")

# Define the columns and primary key for the table
table_definition = CreateTableDefinition(
    columns={
        # This column will store vector embeddings.
        # The NVIDIA integration
        # will automatically generate vector embeddings
        # for any text inserted to this column.
        "example_vector": TableVectorColumnTypeDescriptor(
            dimension=1024, # optional
            service=VectorServiceOptions(
                provider="nvidia",
                model_name="NV-Embed-QA",
            ),
        ),
        # If you want to store the original text
        # in addition to the generated embeddings
        # you must create a separate column.
        "original_text": TableScalarColumnTypeDescriptor(
            column_type=ColumnType.TEXT
        ),
    },
    # Define the primary key for the table.
    # You should change the primary key definition to meet the needs of your data.
    primary_key=TablePrimaryKeyDescriptor(
        partition_by=["original_text"],
        partition_sort={}
    ),
)

table = database.create_table(
    "TABLE_NAME",
    definition=table_definition,
)

print("Created table")

# Index the vector column so that you can perform a vector search on it.
table.create_vector_index(
    "example_vector_index",
    column="example_vector",
    options=TableVectorIndexOptions(
        metric="cosine",
    ),
)

print("Indexed columns")

Use the TypeScript client to create a table with a column that is integrated with NVIDIA:

import { DataAPIClient, Table, InferTableSchema } from '@datastax/astra-db-ts';

const { ASTRA_DB_APPLICATION_TOKEN, ASTRA_DB_API_ENDPOINT } = process.env;

// Initialize the client and get a 'Db' object
const client = new DataAPIClient(ASTRA_DB_APPLICATION_TOKEN);
const db = client.db(ASTRA_DB_API_ENDPOINT);

console.log(`* Connected to DB ${db.id}`);

const tableDefinition = Table.schema({
  // Define all of the columns in the table
  columns: {
    // This column will store vector embeddings.
    // The NVIDIA integration
    // will automatically generate vector embeddings
    // for any text inserted to this column.
    example_vector: {
      type: "vector",
      dimension: 1024, // optional
      service: {
        provider: 'nvidia',
        modelName: 'NV-Embed-QA',
      },
    },
    // If you want to store the original text
    // in addition to the generated embeddings
    // you must create a separate column.
    original_text: "text",
  },
  // Define the primary key for the table.
  // You should change the primary key definition to meet the needs of your data.
  primaryKey: {
    partitionBy: ["original_text"],
  },
});

type TableSchema = InferTableSchema<typeof tableDefinition>;

(async function () {
  const table = await db.createTable<TableSchema>(
    'TABLE_NAME',
    { definition: tableDefinition },
  );

  console.log("Created table");

  // Index the vector column so that you can perform a vector search on it.
  await table.createVectorIndex(
    "example_vector_index",
    "example_vector",
    {
      options: {
        metric: 'cosine',
      },
    },
  );

  console.log("Indexed columns");
})();

Use the Java client to create a table with a column that is integrated with NVIDIA:

import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.core.vector.SimilarityMetric;
import com.datastax.astra.client.core.vectorize.VectorServiceOptions;
import com.datastax.astra.client.databases.Database;
import com.datastax.astra.client.tables.Table;
import com.datastax.astra.client.tables.definition.TableDefinition;
import com.datastax.astra.client.tables.definition.columns.ColumnDefinitionVector;
import com.datastax.astra.client.tables.definition.indexes.TableVectorIndexDefinition;
import com.datastax.astra.client.tables.definition.rows.Row;

import java.util.HashMap;
import java.util.Map;

public class EmbeddingDemo {

  public static void main(String[] args) {
    // Loading Arguments
    String astraToken = System.getenv("ASTRA_DB_APPLICATION_TOKEN");
    String astraApiEndpoint = System.getenv("ASTRA_DB_API_ENDPOINT");

    // Initialize the client
    DataAPIClient client = new DataAPIClient(astraToken);
    System.out.println("Connected to AstraDB");

    Database database = client.getDatabase(astraApiEndpoint);
    System.out.println("Connected to Database.");

    // Define the columns and primary key for the table
    TableDefinition tableDefinition =
        new TableDefinition()
            // This column will store vector embeddings.
            // The NVIDIA integration
            // will automatically generate vector embeddings
            // for any text inserted to this column.
            .addColumnVector(
                "example_vector",
                new ColumnDefinitionVector()
                    .metric(SimilarityMetric.COSINE)
                    .service(
                        new VectorServiceOptions()
                            .provider("nvidia")
                            .modelName("NV-Embed-QA")
                    )
            )
            // If you want to store the original text
            // in addition to the generated embeddings
            // you must create a separate column.
            .addColumnText("original_text")
            // Define the primary key for the table.
            // You should change the primary key definition to meet the needs of your data.
            .addPartitionBy("original_text");

    Table<Row> table = database.createTable("TABLE_NAME", tableDefinition);

    System.out.println("Created table");

    // Index the vector column so that you can perform a vector search on it.
    TableVectorIndexDefinition definition =
        new TableVectorIndexDefinition().column("example_vector").metric(SimilarityMetric.COSINE);

    table.createVectorIndex("example_vector_index", definition);

    System.out.println("Indexed columns");
  }
}

Use the Data API to add the NVIDIA integration to a vector column in a table.

curl -sS -L -X POST "$ASTRA_DB_API_ENDPOINT/api/json/v1/default_keyspace" \
--header "Token: $ASTRA_DB_APPLICATION_TOKEN" \
--header "Content-Type: application/json" \
--data '{
  "createTable": {
    "name": "TABLE_NAME",
    "definition": {
      "columns": {
        # This column will store vector embeddings.
        # The NVIDIA integration
        # will automatically generate vector embeddings
        # for any text inserted to this column.
        "example_vector": {
          "type": "vector",
          "dimension": 1024, # optional
          "service": {
            "provider": "nvidia",
            "modelName": "NV-Embed-QA"
          }
        },
        # If you want to store the original text
        # in addition to the generated embeddings
        # you must create a separate column.
        "original_text": "text"
      },
      # Define the primary key for the table.
      # You should change the primary key definition to meet the needs of your data.
      "primaryKey": "original_text"
    }
  }
}'

Index the vector column so that you can perform a vector search on it.

curl -sS -L -X POST "$ASTRA_DB_API_ENDPOINT/api/json/v1/default_keyspace" \
--header "Token: $ASTRA_DB_APPLICATION_TOKEN" \
--header "Content-Type: application/json" \
--data '{
  "createVectorIndex": {
    "name": "example_vector_index",
    "definition": {
      "column": "example_vector",
      "options": {
        "metric": "cosine",
        "sourceModel": "NV-Embed-QA"
      }
    }
  }
}'

Add the integration to an existing table

If you want to change the embedding provider integration or credential for a column in a table, you alter the table. Use the same column parameters as demonstrated in the previous example.

Automatically generate vector embeddings

After you add the NVIDIA integration to a collection or table, vector embeddings are automatically generated when you insert data.

For collections, vector embeddings are automatically generated when you insert a document with a $vectorize field.

For tables, vector embeddings are automatically generated when you insert a row with a string value for the column that has the NVIDIA integration added.

For more information, see Load your data.

After loading data, you can perform a vector search by providing a natural-language text string. Vectorize generates an embedding from your text string, and then runs the vector search.

Troubleshoot vectorize

When working with vectorize, including the $vectorize reserved field in the Data API, errors can occur from two sources:

Astra DB

There is an issue within Astra DB, including the Astra DB platform, the Data API server, Data API clients, or something else.

Some of the most common Astra DB vectorize errors are related to scoped databases. In your vectorize integration settings, make sure your database is in the scope of the credential that you want to use. Scoped database errors don’t apply to the NVIDIA Astra-hosted embedding provider integration.

When using the Data API with collections, make sure you don’t use $vector and $vectorize in the same query. For more information, see the Data API collections references, such as $vector and $vectorize in collections, Insert many documents, and Sort clauses for collections.

When using the Data API with tables, you can only run a vector search on one vector column at a time. To generate an embedding from a string, the target vector column must have a defined embedding provider integration. For more information, see the Data API tables references, such as Vector type and Sort clauses for tables.

The embedding provider

The embedding provider encountered an issue while processing the embedding generation request. Astra DB passes these errors to you through the Astra Portal or Data API with a qualifying statement such as The embedding provider returned a HTTP client error.

Possible embedding provider errors include rate limiting, billing or account funding issues, and chunk or token size limits. For more information about these errors, see the embedding provider’s documentation, including the documentation for your chosen model.

Carefully read all error messages to determine the source and possible cause for the issue.

NVIDIA token limit

The model for the NVIDIA integration has a token limit of 512. When loading or querying data in collections or tables that use this integration, your $vectorize strings must not exceed this limit.

Was this helpful?

Give Feedback

How can we improve the documentation?

© 2025 DataStax | Privacy policy | Terms of use | Manage Privacy Choices

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Pulsar, Pulsar, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries. Kubernetes is the registered trademark of the Linux Foundation.

General Inquiries: +1 (650) 389-6000, info@datastax.com