Create a vector index
This Astra DB Serverless feature is currently in public preview. Development is ongoing, and the features and functionality are subject to change. Astra DB Serverless, and the use of such, is subject to the DataStax Preview Terms. The Data API tables commands are available through HTTP and the clients. If you use a client, tables commands are available only in client versions 2.0-preview or later. For more information, see Data API client upgrade guide. |
Creates a new index for a vector column in a table in a Serverless (Vector) database. You must create a vector index if you want to perform a vector search on vector embeddings stored in a column.
To create an index on a non-vector column, see Create an index instead.
To manage indexes, your application token must have the same level of permissions that you need to manage tables.
Method signature
-
Python
-
TypeScript
-
Java
-
curl
The following method belongs to the astrapy.table.Table
class.
create_vector_index(
name: str,
*,
column: str,
options: TableVectorIndexOptions | dict[str, Any],
if_not_exists: bool,
table_admin_timeout_ms: int,
request_timeout_ms: int,
timeout_ms: int,
) -> None
The following method belongs to the Table
class.
createVectorIndex(
name: string,
column: WSchema | string,
options?: {
ifNotExists?: boolean,
options?: {
metric?: string,
sourceModel?: string,
timeout?: number | TimeoutDescriptor,
},
}
): Promise<void>
The following methods belong to the com.datastax.astra.client.tables.Table
class.
void createVectorIndex(
String indexName,
String columnName
)
void createVectorIndex(
String indexName,
TableVectorIndexDefinition indexDefinition
)
void createVectorIndex(
String indexName,
TableVectorIndexDefinition indexDefinition,
CreateVectorIndexOptions indexOptions
)
curl -sS -L -X POST "ASTRA_DB_API_ENDPOINT/api/json/v1/ASTRA_DB_KEYSPACE/ASTRA_DB_TABLE" \
--header "Token: ASTRA_DB_APPLICATION_TOKEN" \
--header "Content-Type: application/json" \
--data '{
"createVectorIndex": {
"name": "INDEX_NAME",
"definition": {
"column": "VECTOR_COLUMN_NAME",
"options": {
"metric": STRING,
"sourceModel": STRING
}
}
}
}'
Result
-
Python
-
TypeScript
-
Java
-
curl
Creates an index for the specified vector column.
Does not return anything.
Creates an index for the specified vector column.
Returns a promise that resolves once the operation completes.
Creates an index for the specified vector column.
Does not return anything.
Creates an index for the specified vector column.
If the command succeeds, the response indicates the success.
Example response:
{
"status": {
"ok": 1
}
}
Parameters
-
Python
-
TypeScript
-
Java
-
curl
Name | Type | Summary | ||
---|---|---|---|---|
|
|
The name of the index. Index names must be unique within a keyspace. |
||
|
|
The name of the table column on which to create the index.
The column must be of type To create indexes on non- |
||
|
|
Specifies index options:
If passed, it must be an instance of |
||
|
|
If If
|
||
|
|
A timeout, in milliseconds, to impose on the underlying API request.
If not provided, the |
Name | Type | Summary |
---|---|---|
|
|
The name of the index. Index names must be unique within a keyspace. |
|
|
The name of the table column on which to create the index.
The column must be of type To create indexes on non- |
|
|
The options for this operation. |
Options (TableCreateVectorIndexOptions
):
Name | Type | Summary | ||
---|---|---|---|---|
|
|
If If
|
||
|
|
The similarity metric to use for vector search, one of |
||
|
|
Enable certain vector optimizations on the index by specifying the source model for your vectors, such as |
||
|
|
The client-side timeout for this operation. |
Name | Type | Summary | ||
---|---|---|---|---|
|
|
The name of the index. Index names must be unique within a keyspace. |
||
|
Definition of the index to create. Requires the name of the column to index.
The column must be of type Optionally, you can specify the similarity metric and source model:
|
|||
|
A specialization of index creation options, including If If
|
Name | Type | Summary |
---|---|---|
|
|
The Data API command to create a vector index for a table in a Serverless (Vector) database. It acts as a container for all the attributes and settings required to create the index. |
|
|
The name of the index. Index names must be unique within a keyspace. |
|
|
Contains |
|
|
The name of the table column on which to create the index.
The column must be of type To create indexes on non- |
|
|
Contains either, both, or none of the vector index options:
|
Examples
The following examples demonstrate how to create a vector index.
-
Python
-
TypeScript
-
Java
-
curl
Create a vector index with default settings:
my_table.create_vector_index("m_vector_index", column="m_vector")
Create a vector index and specify the similarity metric for vector searches on this index:
my_table.create_vector_index(
"m_vector_index",
column="m_vector",
options=TableVectorIndexOptions(
metric=VectorMetric.DOT_PRODUCT,
),
)
Example:
Full script
from astrapy import DataAPIClient
client = DataAPIClient("TOKEN")
database = client.get_database("API_ENDPOINT")
from astrapy.constants import SortMode
from astrapy.info import (
CreateTableDefinition,
ColumnType,
)
my_table = database.create_table(
"games",
definition=(
CreateTableDefinition.builder()
.add_column("match_id", ColumnType.TEXT)
.add_column("round", ColumnType.TINYINT)
.add_vector_column("m_vector", dimension=3)
.add_column("score", ColumnType.INT)
.add_column("when", ColumnType.TIMESTAMP)
.add_column("winner", ColumnType.TEXT)
.add_set_column("fighters", ColumnType.UUID)
.add_partition_by(["match_id"])
.add_partition_sort({"round": SortMode.ASCENDING})
.build()
),
)
from astrapy.constants import VectorMetric
from astrapy.info import TableVectorIndexOptions
# create a vector index with dot-product similarity
my_table.create_vector_index(
"m_vector_index",
column="m_vector",
options=TableVectorIndexOptions(
metric=VectorMetric.DOT_PRODUCT,
),
)
# specify a source_model (since the previous statement
# succeeded, this will do nothing because of if_not_exists
):
my_table.create_vector_index(
"m_vector_index",
column="m_vector",
options=TableVectorIndexOptions(
metric=VectorMetric.DOT_PRODUCT,
source_model="nv-qa-4",
),
if_not_exists=True,
)
# leave the settings to the Data API defaults of cosine
# similarity metric (since the previous statement
# succeeded, this will do nothing because of if_not_exists
):
my_table.create_vector_index(
"m_vector_index",
column="m_vector",
if_not_exists=True,
)
from astrapy.constants import VectorMetric
from astrapy.info import TableVectorIndexOptions
# create a vector index with dot-product similarity
my_table.create_vector_index(
"m_vector_index",
column="m_vector",
options=TableVectorIndexOptions(
metric=VectorMetric.DOT_PRODUCT,
),
)
# specify a source_model (since the previous statement
# succeeded, this will do nothing because of if_not_exists
):
my_table.create_vector_index(
"m_vector_index",
column="m_vector",
options=TableVectorIndexOptions(
metric=VectorMetric.DOT_PRODUCT,
source_model="nv-qa-4",
),
if_not_exists=True,
)
# leave the settings to the Data API defaults of cosine
# similarity metric (since the previous statement
# succeeded, this will do nothing because of if_not_exists
):
my_table.create_vector_index(
"m_vector_index",
column="m_vector",
if_not_exists=True,
)
Create a vector index with default settings:
await table.createVectorIndex('m_vector_idx', 'm_vector');
By default, an error occurs if an index with the given name already exists in the keyspace.
To silently ignore existing indexes, use ifNotExists: true
.
If true, an index is created if there is no name collision.
If an index with the given name already exists, the command silently does nothing (neither creates an index nor throws an error).
await table.createVectorIndex('m_vector_idx', 'm_vector', {
ifNotExists: true,
});
Create a vector index and specify the similarity metric for vector searches on this index:
await table.createVectorIndex('m_vector_idx', 'm_vector', {
options: {
metric: 'dot_product',
},
});
Example:
Full script
import { CreateTableDefinition, DataAPIClient, SomeRow } from '@datastax/astra-db-ts';
// Instantiate the client and connect to the database
const client = new DataAPIClient();
const db = client.db(process.env.CLIENT_DB_URL!, { token: process.env.CLIENT_DB_TOKEN! });
// Create table schema using bespoke Data API table definition syntax.
// For information about table definition and data types, see the documentation for createTable.
const TableDefinition = <const>{
columns: {
matchId: 'text'
round: 'tinyint',
mVector: { type: 'vector', dimension: 3 },
score: 'int',
when: 'timestamp',
winner: 'text',
fighters: { type: 'set', valueType: 'uuid' },
},
primaryKey: {
partitionBy: ['matchId'],
partitionSort: { round: 1 },
},
} satisfies CreateTableDefinition;
(async function () {
// Create an untyped table if a 'games' table doesn't already exist
const table = await db.createTable<SomeRow>('games', { definition: TableDefinition, ifNotExists: true });
// Create a secondary index on the 'score' column with default options.
// Errors if a 'score_idx' index already exists in the working keyspace.
await table.createIndex('score_idx', 'score');
// Create a secondary index on the 'winner' column with case-insensitivity
// Because 'ifNotExists: true', the command does not throw an error
// if the working keyspace already has an index named 'winner_idx'.
await table.createIndex('winner_idx', 'winner', {
options: {
caseSensitive: false,
},
ifNotExists: true,
});
// Case insensitive indexes ignore case when querying.
// Insert a row with upper case and lower case characters,
// and then query the row.
// findOne returns a match because 'winner_idx' is case-insensitive.
await table.insertOne({ matchId: '01', round: 0, winner: 'Gray Tist' });
await table.findOne({ winner: 'gray tist' }).then(console.log);
// Create a vector index on the 'mVector' column with cosine similarity (default).
// Errors if an 'm_vector_idx' index already exists in the working keyspace.
await table.createVectorIndex('m_vector_idx', 'mVector');
// Create a vector index on the 'mVector' column with dot-product similarity.
// Because 'ifNotExists: true', the command does not throw an error
// if the working keyspace already has an index named 'm_vector_idx'.
await table.createVectorIndex('m_vector_idx', 'mVector', {
options: {
metric: 'dot_product',
},
ifNotExists: true,
});
// Drop the index so you can recreate it with different options.
await db.dropTableIndex('m_vector_idx');
// Create the vector index with dot-product similarity and a source model.
// For accurate searches, use a source model and metric that are compatible with your vectors.
await table.createVectorIndex('m_vector_idx', 'mVector', {
options: {
metric: 'dot_product',
sourceModel: 'ada002',
},
});
// Vector indexes allow you to perform vector searches.
// Insert a row with a vector, and then run a vector search on the table.
await table.insertOne({ matchId: '01', round: 0, mVector: [0.2, -0.3, -0.5] });
await table.findOne({}, { sort: { mVector: [0.2, -0.3, -0.5] } }).then(console.log);
// Get detailed information about the indexes
// Returns information like [{ name: 'score_idx', definition: { column: 'score', options: {} } }, ...]
await table.listIndexes().then(console.log);
// Get index names only.
// Returns information like ['score_idx', 'winner_idx', 'm_vector_idx', ...]
await table.listIndexes({ nameOnly: true }).then(console.log);
// Drop an index from a database's working keyspace without checking if the index exists.
// If there is no match, the command succeeds but does nothing.
// If there is a match, the named index is deleted.
await db.dropTableIndex('score_idx');
// Drop an index from a database's working keyspace if the index exists.
// Errors if there is no match.
await db.dropTableIndex('score_idx', { ifExists: true });
// Uncomment the following line to drop the table and any related indexes.
// await table.drop();
})();
// Create a vector index on the 'mVector' column with cosine similarity (default).
// Errors if an 'm_vector_idx' index already exists in the working keyspace.
await table.createVectorIndex('m_vector_idx', 'mVector');
// Create a vector index on the 'mVector' column with dot-product similarity.
// Because 'ifNotExists: true', the command does not throw an error
// if the working keyspace already has an index named 'm_vector_idx'.
await table.createVectorIndex('m_vector_idx', 'mVector', {
options: {
metric: 'dot_product',
},
ifNotExists: true,
});
// Drop the index so you can recreate it with different options.
await db.dropTableIndex('m_vector_idx');
// Create the vector index with dot-product similarity and a source model.
// For accurate searches, use a source model and metric that are compatible with your vectors.
await table.createVectorIndex('m_vector_idx', 'mVector', {
options: {
metric: 'dot_product',
sourceModel: 'ada002',
},
});
// Vector indexes allow you to perform vector searches.
// Insert a row with a vector, and then run a vector search on the table.
await table.insertOne({ matchId: '01', round: 0, mVector: [0.2, -0.3, -0.5] });
await table.findOne({}, { sort: { mVector: [0.2, -0.3, -0.5] } }).then(console.log);
Create a vector index with default settings:
// Expects index name and the column to index
tableGames
.createVectorIndex("m_vector_index", "m_vector");
Create a vector index and specify the embedding source model and the similarity metric for vector searches on this index:
TableVectorIndexDefinition definition = new TableVectorIndexDefinition()
.column("m_vector")
.metric(SimilarityMetric.COSINE)
.sourceModel("openai-v3-large");
CreateVectorIndexOptions options = new CreateVectorIndexOptions()
.ifNotExists(true)
.timeout(Duration.ofSeconds(2));
tableGames
.createVectorIndex("m_vector_index", definition, options);
Example:
package com.datastax.astra.client.tables;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.DataAPIClients;
import com.datastax.astra.client.core.vector.SimilarityMetric;
import com.datastax.astra.client.databases.Database;
import com.datastax.astra.client.tables.commands.options.CreateIndexOptions;
import com.datastax.astra.client.tables.commands.options.CreateVectorIndexOptions;
import com.datastax.astra.client.tables.definition.indexes.TableIndexDefinition;
import com.datastax.astra.client.tables.definition.indexes.TableVectorIndexDefinition;
import com.datastax.astra.client.tables.definition.rows.Row;
import java.time.Duration;
public class CreateVectorIndex {
public static void main(String[] args) {
Database db = new DataAPIClient("token").getDatabase("endpoint");
Table<Row> tableGames = db.getTable("games");
//tableGames.createVectorIndex("m_vector_index", "m_vector");
TableVectorIndexDefinition definition = new TableVectorIndexDefinition()
.column("m_vector")
.metric(SimilarityMetric.COSINE)
.sourceModel("openai-v3-large");
CreateVectorIndexOptions options = new CreateVectorIndexOptions()
.ifNotExists(true)
.timeout(Duration.ofSeconds(2));
tableGames.createVectorIndex("m_vector_index", definition, options);
}
}
Create a vector index with default options:
curl -sS -L -X POST "ASTRA_DB_API_ENDPOINT/api/json/v1/ASTRA_DB_KEYSPACE/ASTRA_DB_TABLE" \
--header "Token: ASTRA_DB_APPLICATION_TOKEN" \
--header "Content-Type: application/json" \
--data '{
"createVectorIndex": {
"name": "INDEX_NAME",
"definition": {
"column": "VECTOR_COLUMN_NAME"
}
}
}'
Create a vector index and specify the similarity metric for vector searches on this index:
curl -sS -L -X POST "ASTRA_DB_API_ENDPOINT/api/json/v1/ASTRA_DB_KEYSPACE/ASTRA_DB_TABLE" \
--header "Token: ASTRA_DB_APPLICATION_TOKEN" \
--header "Content-Type: application/json" \
--data '{
"createVectorIndex": {
"name": "INDEX_NAME",
"definition": {
"column": "VECTOR_COLUMN_NAME",
"options": {
"metric": "SIMILARITY_METRIC"
}
}
}
}'
Examples:
curl -sS -L -X POST "ASTRA_DB_API_ENDPOINT/api/json/v1/default_keyspace/students" \
--header "Token: ASTRA_DB_APPLICATION_TOKEN" \
--header "Content-Type: application/json" \
--data '{
"createVectorIndex": {
"name": "index_vector_students",
"definition": {
"column": "vect_emb",
"options": {
"sourceModel": "ada002"
}
}
}
}'
Client reference
-
Python
-
TypeScript
-
Java
-
curl
For more information, see the client reference.
For more information, see the client reference.
For more information, see the client reference.
Client reference documentation is not applicable for HTTP.