Vector database quickstart with Data API

network_check Beginner

query_builder 15 min

Objective

Learn how to create an HCD namespace, connect to your namespace, load a set of vector embeddings using vectorize, and perform a similarity search to find vectors that are close to the one in your query.

Install HCD

Go install HCD if you haven’t already. You’ll also want to install the Data API. For exploration, use the Docker installation with Data API.

Install a terminal to run your client

The clients can be tested by running them in a terminal. You’ll want Xterm, Terminal, or another terminal emulator.

Identify your credentials

You need to identify the credentials, or token, for your database. For initial exploration, you can use the default superuser credentials set in the database. The default superuser credentials are: cassandra is the username and cassandra is the password. These values will be used with a token provider in the client to generate a TOKEN used for authentication to run the client.
1. Before going much further, you should create a new user with a secure password. This new user, once created, will be used to generate a token for authentication.
You need the API endpoint that your namespaces will connect to. The API endpoint is the URL of the database you installed. The port number is 8181 by default. For example, if you installed the database using a Docker container, the API endpoint is http://localhost:8181. This value will be your DB_API_ENDPOINT.
You may either assign your username/password and API endpoint to environment variables in your terminal, or modify the client code to include them directly, as shown in the examples below. Another value that you will want to set is the OPENAI_API_KEY. This is the API key that you received when you signed up for the OpenAI API. This key is used to authenticate your requests to the OpenAI API, and the clients use it to vectorize the text that you provide.
- Linux or macOS
- Windows
- Google Colab
export DB_API_ENDPOINT=DB_API_ENDPOINT # Your database API endpoint export OPENAI_API_KEY=API_KEY # Your OpenAI API key
set DB_DB_API_ENDPOINT=DB_API_ENDPOINT # Your database API endpoint set OPENAI_API_KEY=API_KEY # Your OpenAI API key
import os os.environ["DB_API_ENDPOINT"] = "DB_API_ENDPOINT" # Your database API endpoint os.environ["OPENAI_API_KEY"] = "API_KEY" # Your OpenAI API key

Install the client

Install the library for the language and package manager you’re using.

Python
TypeScript
Java

To install the Python client with pip:

Verify that pip is version 23.0 or higher.
```
pip --version
```
Upgrade pip if needed.
```
python -m pip install --upgrade pip
```
Install the astrapy package. You must have Python 3.8 or higher.
```
pip install astrapy
```

To install the TypeScript client:

Verify that Node is version 14 or higher.
```
node --version
```
Use npm or Yarn to install the TypeScript client.
- npm
- Yarn
To install the TypeScript client with npm:
npm install @datastax/astra-db-ts
To install the TypeScript client with Yarn:
Verify that Yarn is version 2.0 or higher.

yarn --version

Install the astra-db-ts package.

yarn add @datastax/astra-db-ts

Use Maven or Gradle to install the Java client.

Maven
Gradle

To install the Java client with Maven:

Install Java 11+ and Maven 3.9+.

Create a pom.xml file in the root of your project.

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                             http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.example</groupId>
  <artifactId>test-java-client</artifactId>
  <version>1.0-SNAPSHOT</version>

  <!-- The Java client -->
  <dependencies>
    <dependency>
      <groupId>com.datastax.astra</groupId>
      <artifactId>astra-db-java</artifactId>
      <version>1.0.0</version>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>exec-maven-plugin</artifactId>
        <version>3.0.0</version>
        <configuration>
          <executable>java</executable>
          <mainClass>com.example.Quickstart</mainClass>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>java</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>11</source>
          <target>11</target>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

To install the Java client with Gradle:

Install Java 11+ and Gradle.

Create a build.gradle file in the root of your project.

build.gradle

plugins {
    id 'java'
    id 'application'
}

repositories {
    mavenCentral()
}

dependencies {
    implementation 'com.datastax.astra:astra-db-java:1.0.0'
}

application {
    mainClassName = 'com.example.Quickstart'
}

Initialize the client

Paste the following code into a new file on your computer. If you created the environment variables, you don’t need to include the variables in the code.

Python
TypeScript
Java

QuickStartHCD.py

import os
from astrapy import DataAPIClient
from astrapy.constants import Environment
from astrapy.authentication import UsernamePasswordTokenProvider
from astrapy.constants import VectorMetric
from astrapy.ids import UUID
from astrapy.exceptions import InsertManyException

# Database settings
DB_USERNAME = "cassandra"
DB_PASSWORD = "cassandra"
DB_API_ENDPOINT = "http://localhost:8181"
DB_NAMESPACE = "cycling"
DB_COLLECTION = "vector_test"

# Database settings if you exported them as environment variables
# DB_USERNAME = os.environ.get("DB_USERNAME")
# DB_PASSWORD = os.environ.get("DB_PASSWORD")
# DB_API_ENDPOINT = os.environ.get("DB_API_ENDPOINT")

# OpenAI settings
OPEN_AI_PROVIDER = "openai";
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY");
MODEL_NAME = "text-embedding-3-small";

# Build a token
tp = UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD)

# Initialize the client and get a "Database" object
client = DataAPIClient(token=tp, environment=Environment.HCD)
database = client.get_database(DB_API_ENDPOINT, token=tp)

Don’t name the file astrapy.py to avoid a namespace collision.

curl

import { DataAPIClient, UsernamePasswordTokenProvider, VectorDoc, UUID } from '@datastax/astra-db-ts';

// Database settings
const DB_USERNAME = "cassandra";
const DB_PASSWORD = "cassandra";
const DB_API_ENDPOINT = "http://localhost:8181";
const DB_ENVIRONMENT = "hcd";
const DB_NAMESPACE = "cycling";

// Database settings if you exported them as environment variables
// const DB_USERNAME = process.env.DB_USERNAME;
// const DB_PASSWORD = process.env.DB_PASSWORD;
// const DB_API_ENDPOINT = process.env.DB_API_ENDPOINT;

// OpenAI settings
const OPEN_AI_PROVIDER = "openai";
const OPENAI_API_KEY = process.env.OPENAI_API_KEY
const MODEL_NAME = "text-embedding-3-small";

// Build a token in the required format
const tp = new UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD);

// Initialize the client and get a "Db" object
const client = new DataAPIClient({ environment: 'hcd' });
const db = client.db(DB_API_ENDPOINT, { token: tp });
const dbAdmin = db.admin({ environment: 'hcd' });

curl

import com.datastax.astra.client.Collection;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.Database;
import com.datastax.astra.client.admin.DataAPIDatabaseAdmin;
import com.datastax.astra.client.model.CollectionOptions;
import com.datastax.astra.client.model.CommandOptions;
import com.datastax.astra.client.model.Document;
import com.datastax.astra.client.model.FindOneOptions;
import com.datastax.astra.client.model.NamespaceOptions;
import com.datastax.astra.client.model.SimilarityMetric;
import com.datastax.astra.internal.auth.UsernamePasswordTokenProvider;

import java.util.Optional;

import static com.datastax.astra.client.DataAPIClients.DEFAULT_ENDPOINT_LOCAL;
import static com.datastax.astra.client.DataAPIOptions.DataAPIDestination.HCD;
import static com.datastax.astra.client.DataAPIOptions.builder;
import static com.datastax.astra.client.model.Filters.eq;

public class QuickStartHCD {

    public static void main(String[] args) {

        // Database Settings
        String cassandraUserName     = "cassandra";
        String cassandraPassword     = "cassandra";
        String dataApiUrl            = DEFAULT_ENDPOINT_LOCAL;  // http://localhost:8181
        String databaseEnvironment   = "HCD" // DSE, HCD, or ASTRA
        String keyspaceName          = "ks1";
        String collectionName        = "lyrics";

        // Database settings if you export them as environment variables
        // String cassandraUserName            = System.getenv("DB_USERNAME");
        // String cassandraPassword            = System.getenv("DB_PASSWORD");
        // String dataApiUrl                   = System.getenv("DB_API_ENDPOINT");
        
        // OpenAI Embeddings
        String openAiProvider        = "openai";
        String openAiKey             = System.getenv("OPENAI_API_KEY"); // Need to export OPENAI_API_KEY
        String openAiModel           = "text-embedding-3-small";
        int openAiEmbeddingDimension = 1536;

        // Build a token in the form of Cassandra:base64(username):base64(password)
        String token = new UsernamePasswordTokenProvider(cassandraUserName, cassandraPassword).getTokenAsString();
        System.out.println("1/7 - Creating Token: " + token);

        // Initialize the client
        DataAPIClient client = new DataAPIClient(token, builder().withDestination(databaseEnvironment).build());
        System.out.println("2/7 - Connected to Data API");

    }
}

Create a namespace

The clients all support creating a namespace.

Python
TypeScript
Java

QuickStartHCD.py

database.get_database_admin().create_namespace(
    DB_NAMESPACE,
    update_db_namespace=True,
)

Don’t name the file astrapy.py to avoid a namespace collision.

QuickStartHCD.ts

(async () => {
  await dbAdmin.createNamespace(DB_NAMESPACE);
  console.log(await dbAdmin.listNamespaces());
})();

src/main/java/QuickStartHCD.java

        // Create a default keyspace
        ((DataAPIDatabaseAdmin) client
                .getDatabase(dataApiUrl)
                .getDatabaseAdmin()).createNamespace(keyspaceName, NamespaceOptions.simpleStrategy(1));
        System.out.println("3/7 - Keyspace '" + keyspaceName + "'created ");

        Database db = client.getDatabase(dataApiUrl, keyspaceName);
        System.out.println("4/7 - Connected to Database");

Create a collection

Create a collection in your namespace. Choose dimensions that match your vector data and pick an appropriate similarity metric: cosine (default), dot_product, or euclidean.

The embeddings will be generated using the vectorize method, so the collection needs the parameters for using an embedding service.

Python
TypeScript
Java

QuickStartHCD.py

import os
from astrapy import DataAPIClient
from astrapy.constants import Environment
from astrapy.authentication import UsernamePasswordTokenProvider
from astrapy.constants import VectorMetric
from astrapy.ids import UUID
from astrapy.exceptions import InsertManyException

# Database settings
DB_USERNAME = "cassandra"
DB_PASSWORD = "cassandra"
DB_API_ENDPOINT = "http://localhost:8181"
DB_NAMESPACE = "cycling"
DB_COLLECTION = "vector_test"

# Database settings if you exported them as environment variables
# DB_USERNAME = os.environ.get("DB_USERNAME")
# DB_PASSWORD = os.environ.get("DB_PASSWORD")
# DB_API_ENDPOINT = os.environ.get("DB_API_ENDPOINT")

# OpenAI settings
OPEN_AI_PROVIDER = "openai";
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY");
MODEL_NAME = "text-embedding-3-small";

# Build a token
tp = UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD)

# Initialize the client and get a "Database" object
client = DataAPIClient(token=tp, environment=Environment.HCD)
database = client.get_database(DB_API_ENDPOINT, token=tp)

# ⬇️ NEW CODE

# Create a collection. The default similarity metric is cosine. If you're not
# sure what dimension to set, use whatever dimension vector your embeddings
# model produces.
collection = database.create_collection(
    DB_COLLECTION,
    dimension=5,
    metric=VectorMetric.COSINE,  # or simply "cosine"
    service={"provider": OPEN_AI_PROVIDER, "modelName": MODEL_NAME},
        embedding_api_key=OPENAI_API_KEY,
    namespace=DB_NAMESPACE,
    check_exists=False,
)
print(f"* Collection: {collection.full_name}\n")

# ⬆️ NEW CODE

QuickStartHCD.ts

import { DataAPIClient, UsernamePasswordTokenProvider, VectorDoc, UUID } from '@datastax/astra-db-ts';

// Database settings
const DB_USERNAME = "cassandra";
const DB_PASSWORD = "cassandra";
const DB_API_ENDPOINT = "http://localhost:8181";
const DB_ENVIRONMENT = "hcd";
const DB_NAMESPACE = "cycling";

// Database settings if you exported them as environment variables
// const DB_USERNAME = process.env.DB_USERNAME;
// const DB_PASSWORD = process.env.DB_PASSWORD;
// const DB_API_ENDPOINT = process.env.DB_API_ENDPOINT;

// OpenAI settings
const OPEN_AI_PROVIDER = "openai";
const OPENAI_API_KEY = process.env.OPENAI_API_KEY
const MODEL_NAME = "text-embedding-3-small";

// Build a token in the required format
const tp = new UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD);

// Initialize the client and get a "Db" object
const client = new DataAPIClient({ environment: 'hcd' });
const db = client.db(DB_API_ENDPOINT, { token: tp });
const dbAdmin = db.admin({ environment: 'hcd' });

// ⬇️ NEW CODE

// Schema for the collection (VectorDoc adds the $vector field)
interface Idea extends VectorDoc {
  idea: string,
}

(async function () {
  // Create a typed, vector-enabled collection. The default metric is cosine.
  // If you're not sure what dimension to set, use whatever dimension vector
  // your embeddings model produces.
  const collection = await db.createCollection<Idea>('vector_test', {
    namespace: DB_NAMESPACE,
    vector: {
      service: {
        provider: OPEN_AI_PROVIDER,
        modelName: MODEL_NAME
      },
      dimension: 5,
      metric: 'cosine',
    },
    embeddingApiKey: OPENAI_API_KEY,
    checkExists: false
  });
  console.log(`* Created collection ${collection.namespace}.${collection.collectionName}`);
})();

// ⬆️ NEW CODE

src/main/java/com/example/QuickStartHCD.java

import com.datastax.astra.client.Collection;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.Database;
import com.datastax.astra.client.admin.DataAPIDatabaseAdmin;
import com.datastax.astra.client.model.CollectionOptions;
import com.datastax.astra.client.model.CommandOptions;
import com.datastax.astra.client.model.Document;
import com.datastax.astra.client.model.FindOneOptions;
import com.datastax.astra.client.model.NamespaceOptions;
import com.datastax.astra.client.model.SimilarityMetric;
import com.datastax.astra.internal.auth.UsernamePasswordTokenProvider;

import java.util.Optional;

import static com.datastax.astra.client.DataAPIClients.DEFAULT_ENDPOINT_LOCAL;
import static com.datastax.astra.client.DataAPIOptions.DataAPIDestination.HCD;
import static com.datastax.astra.client.DataAPIOptions.builder;
import static com.datastax.astra.client.model.Filters.eq;

public class QuickStartHCD {

    public static void main(String[] args) {

        // Database Settings
        String cassandraUserName     = "cassandra";
        String cassandraPassword     = "cassandra";
        String dataApiUrl            = DEFAULT_ENDPOINT_LOCAL;  // http://localhost:8181
        String databaseEnvironment   = "HCD" // DSE, HCD, or ASTRA
        String keyspaceName          = "ks1";
        String collectionName        = "lyrics";

        // Database settings if you export them as environment variables
        // String cassandraUserName            = System.getenv("DB_USERNAME");
        // String cassandraPassword            = System.getenv("DB_PASSWORD");
        // String dataApiUrl                   = System.getenv("DB_API_ENDPOINT");
        
        // OpenAI Embeddings
        String openAiProvider        = "openai";
        String openAiKey             = System.getenv("OPENAI_API_KEY"); // Need to export OPENAI_API_KEY
        String openAiModel           = "text-embedding-3-small";
        int openAiEmbeddingDimension = 1536;

        // Build a token in the form of Cassandra:base64(username):base64(password)
        String token = new UsernamePasswordTokenProvider(cassandraUserName, cassandraPassword).getTokenAsString();
        System.out.println("1/7 - Creating Token: " + token);

        // Initialize the client
        DataAPIClient client = new DataAPIClient(token, builder().withDestination(databaseEnvironment).build());
        System.out.println("2/7 - Connected to Data API");

    // ⬇️ NEW CODE

        // Create a collection 
        Collection<Document> collectionLyrics =  db.createCollection(collectionName, CollectionOptions.builder()
        .vectorDimension(5)
        .vectorSimilarity(SimilarityMetric.COSINE)
        .build(),
        System.out.println("5/7 - Collection created");

    // ⬆️ NEW CODE

    }
}

Load vector embeddings

Insert a few documents into the collection. Two methods are available for inserting data: $vectorize and $vector. The $vectorize method generate embeddings using a specified embedding service. The $vector method is used when you already have embeddings.

Use the `$vectorize` method

Python
TypeScript
Java

QuickStartHCD.py

import os
from astrapy import DataAPIClient
from astrapy.constants import Environment
from astrapy.authentication import UsernamePasswordTokenProvider
from astrapy.constants import VectorMetric
from astrapy.ids import UUID
from astrapy.exceptions import InsertManyException

# Database settings
DB_USERNAME = "cassandra"
DB_PASSWORD = "cassandra"
DB_API_ENDPOINT = "http://localhost:8181"
DB_NAMESPACE = "cycling"
DB_COLLECTION = "vector_test"

# Database settings if you exported them as environment variables
# DB_USERNAME = os.environ.get("DB_USERNAME")
# DB_PASSWORD = os.environ.get("DB_PASSWORD")
# DB_API_ENDPOINT = os.environ.get("DB_API_ENDPOINT")

# OpenAI settings
OPEN_AI_PROVIDER = "openai";
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY");
MODEL_NAME = "text-embedding-3-small";

# Build a token
tp = UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD)

# Initialize the client and get a "Database" object
client = DataAPIClient(token=tp, environment=Environment.HCD)
database = client.get_database(DB_API_ENDPOINT, token=tp)

# Create a collection. The default similarity metric is cosine. If you're not
# sure what dimension to set, use whatever dimension vector your embeddings
# model produces.
collection = database.create_collection(
    DB_COLLECTION,
    dimension=5,
    metric=VectorMetric.COSINE,  # or simply "cosine"
    service={"provider": OPEN_AI_PROVIDER, "modelName": MODEL_NAME},
        embedding_api_key=OPENAI_API_KEY,
    namespace=DB_NAMESPACE,
    check_exists=False,
)
print(f"* Collection: {collection.full_name}\n")

# ⬇️ NEW CODE

# Insert documents into the collection.
# (UUIDs here are version 7.)
documents = [
    {
        "_id": UUID("018e65c9-df45-7913-89f8-175f28bd7f74"),
        "text": "Chatbot integrated sneakers that talk to you",
        "$vectorize": "Wild! How can they do that?"
    },
    {
        "_id": UUID("018e65c9-e1b7-7048-a593-db452be1e4c2"),
        "text": "An AI quilt to help you sleep forever",
        "$vectorize": "Sleep like a baby soft and cuddly"
    },
    {
        "_id": UUID("018e65c9-e33d-749b-9386-e848739582f0"),
        "text": "A deep learning display that controls your mood",
        "$vectorize": "I do not want my mood controlled!"
    },
]
try:
    insertion_result = collection.insert_many(documents)
    print(f"* Inserted {len(insertion_result.inserted_ids)} items.\n")
except InsertManyException:
    print("* Documents found on DB already. Let's move on.\n")

# ⬆️ NEW CODE

QuickStartHCD.ts

import { DataAPIClient, UsernamePasswordTokenProvider, VectorDoc, UUID } from '@datastax/astra-db-ts';

// Database settings
const DB_USERNAME = "cassandra";
const DB_PASSWORD = "cassandra";
const DB_API_ENDPOINT = "http://localhost:8181";
const DB_ENVIRONMENT = "hcd";
const DB_NAMESPACE = "cycling";

// Database settings if you exported them as environment variables
// const DB_USERNAME = process.env.DB_USERNAME;
// const DB_PASSWORD = process.env.DB_PASSWORD;
// const DB_API_ENDPOINT = process.env.DB_API_ENDPOINT;

// OpenAI settings
const OPEN_AI_PROVIDER = "openai";
const OPENAI_API_KEY = process.env.OPENAI_API_KEY
const MODEL_NAME = "text-embedding-3-small";

// Build a token in the required format
const tp = new UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD);

// Initialize the client and get a "Db" object
const client = new DataAPIClient({ environment: 'hcd' });
const db = client.db(DB_API_ENDPOINT, { token: tp });
const dbAdmin = db.admin({ environment: 'hcd' });

// Schema for the collection (VectorDoc adds the $vector field)
interface Idea extends VectorDoc {
  idea: string,
}

(async function () {
  // Create a typed, vector-enabled collection. The default metric is cosine.
  // If you're not sure what dimension to set, use whatever dimension vector
  // your embeddings model produces.
  const collection = await db.createCollection<Idea>('vector_test', {
    namespace: DB_NAMESPACE,
    vector: {
      service: {
        provider: OPEN_AI_PROVIDER,
        modelName: MODEL_NAME
      },
      dimension: 5,
      metric: 'cosine',
    },
    embeddingApiKey: OPENAI_API_KEY,
    checkExists: false
  });
  console.log(`* Created collection ${collection.namespace}.${collection.collectionName}`);

  // ⬇️ NEW CODE

  // Insert documents into the collection (using UUIDv7s)
  const documents = [
    {
      _id: new UUID('018e65c9-df45-7913-89f8-175f28bd7f74'),
      text: 'ChatGPT integrated sneakers that talk to you',
      $vectorize: 'Wild! How can they do that?',
    },
    {
      _id: new UUID('018e65c9-e1b7-7048-a593-db452be1e4c2'),
      text: 'An AI quilt to help you sleep forever',
      $vectorize: 'Sleep like a baby soft and cuddly',
    },
    {
      _id: new UUID('018e65c9-e33d-749b-9386-e848739582f0'),
      text: 'A deep learning display that controls your mood',
      $vectorize: 'I do not want my mood controlled!',
    },
  ];

  try {
    const inserted = await collection.insertMany(documents);
    console.log(`* Inserted ${inserted.insertedCount} items.`);
  } catch (e) {
    console.log('* Documents found on DB already. Let\'s move on!');
  }

  // ⬆️ NEW CODE
})();

src/main/java/com/example/QuickStartHCD.java

import com.datastax.astra.client.Collection;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.Database;
import com.datastax.astra.client.admin.DataAPIDatabaseAdmin;
import com.datastax.astra.client.model.CollectionOptions;
import com.datastax.astra.client.model.CommandOptions;
import com.datastax.astra.client.model.Document;
import com.datastax.astra.client.model.FindOneOptions;
import com.datastax.astra.client.model.NamespaceOptions;
import com.datastax.astra.client.model.SimilarityMetric;
import com.datastax.astra.internal.auth.UsernamePasswordTokenProvider;

import java.util.Optional;

import static com.datastax.astra.client.DataAPIClients.DEFAULT_ENDPOINT_LOCAL;
import static com.datastax.astra.client.DataAPIOptions.DataAPIDestination.HCD;
import static com.datastax.astra.client.DataAPIOptions.builder;
import static com.datastax.astra.client.model.Filters.eq;

public class QuickStartHCD {

    public static void main(String[] args) {

        // Database Settings
        String cassandraUserName     = "cassandra";
        String cassandraPassword     = "cassandra";
        String dataApiUrl            = DEFAULT_ENDPOINT_LOCAL;  // http://localhost:8181
        String databaseEnvironment   = "HCD" // DSE, HCD, or ASTRA
        String keyspaceName          = "ks1";
        String collectionName        = "lyrics";

        // Database settings if you export them as environment variables
        // String cassandraUserName            = System.getenv("DB_USERNAME");
        // String cassandraPassword            = System.getenv("DB_PASSWORD");
        // String dataApiUrl                   = System.getenv("DB_API_ENDPOINT");
        
        // OpenAI Embeddings
        String openAiProvider        = "openai";
        String openAiKey             = System.getenv("OPENAI_API_KEY"); // Need to export OPENAI_API_KEY
        String openAiModel           = "text-embedding-3-small";
        int openAiEmbeddingDimension = 1536;

        // Build a token in the form of Cassandra:base64(username):base64(password)
        String token = new UsernamePasswordTokenProvider(cassandraUserName, cassandraPassword).getTokenAsString();
        System.out.println("1/7 - Creating Token: " + token);

        // Initialize the client
        DataAPIClient client = new DataAPIClient(token, builder().withDestination(databaseEnvironment).build());
        System.out.println("2/7 - Connected to Data API");

        // Create a collection 
        Collection<Document> collectionLyrics =  db.createCollection(collectionName, CollectionOptions.builder()
        .vectorDimension(5)
        .vectorSimilarity(SimilarityMetric.COSINE)
        .build(),
        System.out.println("5/7 - Collection created");

    // ⬇️ NEW CODE

    // Insert some documents
    collectionLyrics.insertMany(
        new Document(1).append("band", "Dire Straits").append("song", "Romeo And Juliet").vectorize("A lovestruck Romeo sings the streets a serenade"),
        new Document(2).append("band", "Dire Straits").append("song", "Romeo And Juliet").vectorize("Says something like, You and me babe, how about it?"),
        new Document(4).append("band", "Dire Straits").append("song", "Romeo And Juliet").vectorize("Juliet says,Hey, it's Romeo, you nearly gimme a heart attack"),
        new Document(5).append("band", "Dire Straits").append("song", "Romeo And Juliet").vectorize("He's underneath the window"),
        new Document(6).append("band", "Dire Straits").append("song", "Romeo And Juliet").vectorize("She's singing, Hey la, my boyfriend's back"),
        new Document(7).append("band", "Dire Straits").append("song", "Romeo And Juliet").vectorize("You shouldn't come around here singing up at people like that"),
        new Document(8).append("band", "Dire Straits").append("song", "Romeo And Juliet").vectorize("Anyway, what you gonna do about it?"));
    System.out.println("6/7 - Collection populated");

    // ⬆️ NEW CODE

    }
}

Use the `$vector` method instead of `$vectorize`

The $vector method can be used if you already have embeddings.

Python
TypeScript
Java

QuickStartHCD.py

import os
from astrapy import DataAPIClient
from astrapy.constants import Environment
from astrapy.authentication import UsernamePasswordTokenProvider
from astrapy.constants import VectorMetric
from astrapy.ids import UUID
from astrapy.exceptions import InsertManyException

# Database settings
DB_USERNAME = "cassandra"
DB_PASSWORD = "cassandra"
DB_API_ENDPOINT = "http://localhost:8181"
DB_NAMESPACE = "cycling"
DB_COLLECTION = "vector_test"

# Database settings if you exported them as environment variables
# DB_USERNAME = os.environ.get("DB_USERNAME")
# DB_PASSWORD = os.environ.get("DB_PASSWORD")
# DB_API_ENDPOINT = os.environ.get("DB_API_ENDPOINT")

# OpenAI settings
OPEN_AI_PROVIDER = "openai";
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY");
MODEL_NAME = "text-embedding-3-small";

# Build a token
tp = UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD)

# Initialize the client and get a "Database" object
client = DataAPIClient(token=tp, environment=Environment.HCD)
database = client.get_database(DB_API_ENDPOINT, token=tp)

# Create a collection. The default similarity metric is cosine. If you're not
# sure what dimension to set, use whatever dimension vector your embeddings
# model produces.
collection = database.create_collection(
    DB_COLLECTION,
    dimension=5,
    metric=VectorMetric.COSINE,  # or simply "cosine"
    service={"provider": OPEN_AI_PROVIDER, "modelName": MODEL_NAME},
        embedding_api_key=OPENAI_API_KEY,
    namespace=DB_NAMESPACE,
    check_exists=False,
)
print(f"* Collection: {collection.full_name}\n")

# ⬇️ NEW CODE

# Insert documents into the collection.
# (UUIDs here are version 7.)
documents = [
    {
        "_id": UUID("018e65c9-df45-7913-89f8-175f28bd7f74"),
        "text": "Chatbot integrated sneakers that talk to you",
        "$vector": [0.1, 0.15, 0.3, 0.12, 0.05],
    },
    {
        "_id": UUID("018e65c9-e1b7-7048-a593-db452be1e4c2"),
        "text": "An AI quilt to help you sleep forever",
        "$vector": [0.45, 0.09, 0.01, 0.2, 0.11],
    },
    {
        "_id": UUID("018e65c9-e33d-749b-9386-e848739582f0"),
        "text": "A deep learning display that controls your mood",
        "$vector": [0.1, 0.05, 0.08, 0.3, 0.6],
    },
]
try:
    insertion_result = collection.insert_many(documents)
    print(f"* Inserted {len(insertion_result.inserted_ids)} items.\n")
except InsertManyException:
    print("* Documents found on DB already. Let's move on.\n")

# ⬆️ NEW CODE

QuickStartHCD.ts

import { DataAPIClient, UsernamePasswordTokenProvider, VectorDoc, UUID } from '@datastax/astra-db-ts';

// Database settings
const DB_USERNAME = "cassandra";
const DB_PASSWORD = "cassandra";
const DB_API_ENDPOINT = "http://localhost:8181";
const DB_ENVIRONMENT = "hcd";
const DB_NAMESPACE = "cycling";

// Database settings if you exported them as environment variables
// const DB_USERNAME = process.env.DB_USERNAME;
// const DB_PASSWORD = process.env.DB_PASSWORD;
// const DB_API_ENDPOINT = process.env.DB_API_ENDPOINT;

// OpenAI settings
const OPEN_AI_PROVIDER = "openai";
const OPENAI_API_KEY = process.env.OPENAI_API_KEY
const MODEL_NAME = "text-embedding-3-small";

// Build a token in the required format
const tp = new UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD);

// Initialize the client and get a "Db" object
const client = new DataAPIClient({ environment: 'hcd' });
const db = client.db(DB_API_ENDPOINT, { token: tp });
const dbAdmin = db.admin({ environment: 'hcd' });

// Schema for the collection (VectorDoc adds the $vector field)
interface Idea extends VectorDoc {
  idea: string,
}

(async function () {
  // Create a typed, vector-enabled collection. The default metric is cosine.
  // If you're not sure what dimension to set, use whatever dimension vector
  // your embeddings model produces.
  const collection = await db.createCollection<Idea>('vector_test', {
    namespace: DB_NAMESPACE,
    vector: {
      service: {
        provider: OPEN_AI_PROVIDER,
        modelName: MODEL_NAME
      },
      dimension: 5,
      metric: 'cosine',
    },
    embeddingApiKey: OPENAI_API_KEY,
    checkExists: false
  });
  console.log(`* Created collection ${collection.namespace}.${collection.collectionName}`);

  // ⬇️ NEW CODE

  // Insert documents into the collection (using UUIDv7s)
  const documents = [
    {
      _id: new UUID('018e65c9-df45-7913-89f8-175f28bd7f74'),
      text: 'ChatGPT integrated sneakers that talk to you',
      $vector: [0.25, 0.25, 0.25, 0.25, 0.45],
    },
    {
      _id: new UUID('018e65c9-e1b7-7048-a593-db452be1e4c2'),
      text: 'An AI quilt to help you sleep forever',
      $vector: [0.10, 0.15, 0.25, 0.25, 0.15],
    },
    {
      _id: new UUID('018e65c9-e33d-749b-9386-e848739582f0'),
      text: 'A deep learning display that controls your mood',
      $vector: 'I do not want my mood controlled!',
    },
  ];

  try {
    const inserted = await collection.insertMany(documents);
    console.log(`* Inserted ${inserted.insertedCount} items.`);
  } catch (e) {
    console.log('* Documents found on DB already. Let\'s move on!');
  }

  // ⬆️ NEW CODE
})();

src/main/java/com/example/QuickStartHCD.java

import com.datastax.astra.client.Collection;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.Database;
import com.datastax.astra.client.admin.DataAPIDatabaseAdmin;
import com.datastax.astra.client.model.CollectionOptions;
import com.datastax.astra.client.model.CommandOptions;
import com.datastax.astra.client.model.Document;
import com.datastax.astra.client.model.FindOneOptions;
import com.datastax.astra.client.model.NamespaceOptions;
import com.datastax.astra.client.model.SimilarityMetric;
import com.datastax.astra.internal.auth.UsernamePasswordTokenProvider;

import java.util.Optional;

import static com.datastax.astra.client.DataAPIClients.DEFAULT_ENDPOINT_LOCAL;
import static com.datastax.astra.client.DataAPIOptions.DataAPIDestination.HCD;
import static com.datastax.astra.client.DataAPIOptions.builder;
import static com.datastax.astra.client.model.Filters.eq;

public class QuickStartHCD {

    public static void main(String[] args) {

        // Database Settings
        String cassandraUserName     = "cassandra";
        String cassandraPassword     = "cassandra";
        String dataApiUrl            = DEFAULT_ENDPOINT_LOCAL;  // http://localhost:8181
        String databaseEnvironment   = "HCD" // DSE, HCD, or ASTRA
        String keyspaceName          = "ks1";
        String collectionName        = "lyrics";

        // Database settings if you export them as environment variables
        // String cassandraUserName            = System.getenv("DB_USERNAME");
        // String cassandraPassword            = System.getenv("DB_PASSWORD");
        // String dataApiUrl                   = System.getenv("DB_API_ENDPOINT");
        
        // OpenAI Embeddings
        String openAiProvider        = "openai";
        String openAiKey             = System.getenv("OPENAI_API_KEY"); // Need to export OPENAI_API_KEY
        String openAiModel           = "text-embedding-3-small";
        int openAiEmbeddingDimension = 1536;

        // Build a token in the form of Cassandra:base64(username):base64(password)
        String token = new UsernamePasswordTokenProvider(cassandraUserName, cassandraPassword).getTokenAsString();
        System.out.println("1/7 - Creating Token: " + token);

        // Initialize the client
        DataAPIClient client = new DataAPIClient(token, builder().withDestination(databaseEnvironment).build());
        System.out.println("2/7 - Connected to Data API");

        // Create a collection 
        Collection<Document> collectionLyrics =  db.createCollection(collectionName, CollectionOptions.builder()
        .vectorDimension(5)
        .vectorSimilarity(SimilarityMetric.COSINE)
        .build(),
        System.out.println("5/7 - Collection created");

    // ⬇️ NEW CODE

    // Insert some documents
    collection.insertMany(
        new Document("1")
                .append("text", "ChatGPT integrated sneakers that talk to you")
                .vector(new float[]{0.1f, 0.15f, 0.3f, 0.12f, 0.05f}),
        new Document("2")
                .append("text", "An AI quilt to help you sleep forever")
                .vector(new float[]{0.45f, 0.09f, 0.01f, 0.2f, 0.11f}),
        new Document("3")
                .append("text", "A deep learning display that controls your mood")
                .vector(new float[]{0.1f, 0.05f, 0.08f, 0.3f, 0.6f}));
    System.out.println("6/7 - Collection populated");

    // ⬆️ NEW CODE

    }
}

Perform a similarity search

Find documents that are close to a specific vector embedding. (The code also shows the optional step of dropping the collection at the end.)

Python
TypeScript
Java

QuickStartHCD.py

import os
from astrapy import DataAPIClient
from astrapy.constants import Environment
from astrapy.authentication import UsernamePasswordTokenProvider
from astrapy.constants import VectorMetric
from astrapy.ids import UUID
from astrapy.exceptions import InsertManyException

# Database settings
DB_USERNAME = "cassandra"
DB_PASSWORD = "cassandra"
DB_API_ENDPOINT = "http://localhost:8181"
DB_NAMESPACE = "cycling"
DB_COLLECTION = "vector_test"

# Database settings if you exported them as environment variables
# DB_USERNAME = os.environ.get("DB_USERNAME")
# DB_PASSWORD = os.environ.get("DB_PASSWORD")
# DB_API_ENDPOINT = os.environ.get("DB_API_ENDPOINT")

# OpenAI settings
OPEN_AI_PROVIDER = "openai";
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY");
MODEL_NAME = "text-embedding-3-small";

# Build a token
tp = UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD)

# Initialize the client and get a "Database" object
client = DataAPIClient(token=tp, environment=Environment.HCD)
database = client.get_database(DB_API_ENDPOINT, token=tp)

# Create a collection. The default similarity metric is cosine. If you're not
# sure what dimension to set, use whatever dimension vector your embeddings
# model produces.
collection = database.create_collection(
    DB_COLLECTION,
    dimension=5,
    metric=VectorMetric.COSINE,  # or simply "cosine"
    service={"provider": OPEN_AI_PROVIDER, "modelName": MODEL_NAME},
        embedding_api_key=OPENAI_API_KEY,
    namespace=DB_NAMESPACE,
    check_exists=False,
)
print(f"* Collection: {collection.full_name}\n")

# Insert documents into the collection.
# (UUIDs here are version 7.)
documents = [
    {
        "_id": UUID("018e65c9-df45-7913-89f8-175f28bd7f74"),
        "text": "Chatbot integrated sneakers that talk to you",
        "$vector": [0.1, 0.15, 0.3, 0.12, 0.05],
    },
    {
        "_id": UUID("018e65c9-e1b7-7048-a593-db452be1e4c2"),
        "text": "An AI quilt to help you sleep forever",
        "$vector": [0.45, 0.09, 0.01, 0.2, 0.11],
    },
    {
        "_id": UUID("018e65c9-e33d-749b-9386-e848739582f0"),
        "text": "A deep learning display that controls your mood",
        "$vector": [0.1, 0.05, 0.08, 0.3, 0.6],
    },
]
try:
    insertion_result = collection.insert_many(documents)
    print(f"* Inserted {len(insertion_result.inserted_ids)} items.\n")
except InsertManyException:
    print("* Documents found on DB already. Let's move on.\n")

# ⬇️ NEW CODE

# Perform a similarity search
query = [0.15, 0.1, 0.1, 0.35, 0.55]
results = collection.find(
    vector=query,
    limit=10,
)
print("Vector search results:")
for document in results:
    print("    ", document)

# ⬆️ NEW CODE

QuickStartHCD.ts

import { DataAPIClient, UsernamePasswordTokenProvider, VectorDoc, UUID } from '@datastax/astra-db-ts';

// Database settings
const DB_USERNAME = "cassandra";
const DB_PASSWORD = "cassandra";
const DB_API_ENDPOINT = "http://localhost:8181";
const DB_ENVIRONMENT = "hcd";
const DB_NAMESPACE = "cycling";

// Database settings if you exported them as environment variables
// const DB_USERNAME = process.env.DB_USERNAME;
// const DB_PASSWORD = process.env.DB_PASSWORD;
// const DB_API_ENDPOINT = process.env.DB_API_ENDPOINT;

// OpenAI settings
const OPEN_AI_PROVIDER = "openai";
const OPENAI_API_KEY = process.env.OPENAI_API_KEY
const MODEL_NAME = "text-embedding-3-small";

// Build a token in the required format
const tp = new UsernamePasswordTokenProvider(DB_USERNAME, DB_PASSWORD);

// Initialize the client and get a "Db" object
const client = new DataAPIClient({ environment: 'hcd' });
const db = client.db(DB_API_ENDPOINT, { token: tp });
const dbAdmin = db.admin({ environment: 'hcd' });

// Schema for the collection (VectorDoc adds the $vector field)
interface Idea extends VectorDoc {
  idea: string,
}

(async function () {
  // Create a typed, vector-enabled collection. The default metric is cosine.
  // If you're not sure what dimension to set, use whatever dimension vector
  // your embeddings model produces.
  const collection = await db.createCollection<Idea>('vector_test', {
    namespace: DB_NAMESPACE,
    vector: {
      service: {
        provider: OPEN_AI_PROVIDER,
        modelName: MODEL_NAME
      },
      dimension: 5,
      metric: 'cosine',
    },
    embeddingApiKey: OPENAI_API_KEY,
    checkExists: false
  });
  console.log(`* Created collection ${collection.namespace}.${collection.collectionName}`);

  // Insert documents into the collection (using UUIDv7s)
  const documents = [
    {
      _id: new UUID('018e65c9-df45-7913-89f8-175f28bd7f74'),
      text: 'ChatGPT integrated sneakers that talk to you',
      $vector: [0.25, 0.25, 0.25, 0.25, 0.45],
    },
    {
      _id: new UUID('018e65c9-e1b7-7048-a593-db452be1e4c2'),
      text: 'An AI quilt to help you sleep forever',
      $vector: [0.10, 0.15, 0.25, 0.25, 0.15],
    },
    {
      _id: new UUID('018e65c9-e33d-749b-9386-e848739582f0'),
      text: 'A deep learning display that controls your mood',
      $vector: 'I do not want my mood controlled!',
    },
  ];

  try {
    const inserted = await collection.insertMany(documents);
    console.log(`* Inserted ${inserted.insertedCount} items.`);
  } catch (e) {
    console.log('* Documents found on DB already. Let\'s move on!');
  }

  // ⬇️ NEW CODE

  // Perform a similarity search
  const cursor = await collection.find({}, {
    vector: [0.15, 0.1, 0.1, 0.35, 0.55],
    limit: 10,
    includeSimilarity: true,
  });

  console.log('* Search results:')
  for await (const doc of cursor) {
    console.log('  ', doc.text, doc.$similarity);
  }

  // Cleanup (if desired)
  await db.dropCollection('vector_test');
  console.log('* Collection dropped.');

  // Close the client
  await client.close();

  // ⬆️ NEW CODE
})();

src/main/java/com/example/QuickStartHCD.java

import com.datastax.astra.client.Collection;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.Database;
import com.datastax.astra.client.admin.DataAPIDatabaseAdmin;
import com.datastax.astra.client.model.CollectionOptions;
import com.datastax.astra.client.model.CommandOptions;
import com.datastax.astra.client.model.Document;
import com.datastax.astra.client.model.FindOneOptions;
import com.datastax.astra.client.model.NamespaceOptions;
import com.datastax.astra.client.model.SimilarityMetric;
import com.datastax.astra.internal.auth.UsernamePasswordTokenProvider;

import java.util.Optional;

import static com.datastax.astra.client.DataAPIClients.DEFAULT_ENDPOINT_LOCAL;
import static com.datastax.astra.client.DataAPIOptions.DataAPIDestination.HCD;
import static com.datastax.astra.client.DataAPIOptions.builder;
import static com.datastax.astra.client.model.Filters.eq;

public class QuickStartHCD {

    public static void main(String[] args) {

        // Database Settings
        String cassandraUserName     = "cassandra";
        String cassandraPassword     = "cassandra";
        String dataApiUrl            = DEFAULT_ENDPOINT_LOCAL;  // http://localhost:8181
        String databaseEnvironment   = "HCD" // DSE, HCD, or ASTRA
        String keyspaceName          = "ks1";
        String collectionName        = "lyrics";

        // Database settings if you export them as environment variables
        // String cassandraUserName            = System.getenv("DB_USERNAME");
        // String cassandraPassword            = System.getenv("DB_PASSWORD");
        // String dataApiUrl                   = System.getenv("DB_API_ENDPOINT");
        
        // OpenAI Embeddings
        String openAiProvider        = "openai";
        String openAiKey             = System.getenv("OPENAI_API_KEY"); // Need to export OPENAI_API_KEY
        String openAiModel           = "text-embedding-3-small";
        int openAiEmbeddingDimension = 1536;

        // Build a token in the form of Cassandra:base64(username):base64(password)
        String token = new UsernamePasswordTokenProvider(cassandraUserName, cassandraPassword).getTokenAsString();
        System.out.println("1/7 - Creating Token: " + token);

        // Initialize the client
        DataAPIClient client = new DataAPIClient(token, builder().withDestination(databaseEnvironment).build());
        System.out.println("2/7 - Connected to Data API");

        // Create a collection 
        Collection<Document> collectionLyrics =  db.createCollection(collectionName, CollectionOptions.builder()
        .vectorDimension(5)
        .vectorSimilarity(SimilarityMetric.COSINE)
        .build(),
        System.out.println("5/7 - Collection created");

    // Insert some documents
    collection.insertMany(
        new Document("1")
                .append("text", "ChatGPT integrated sneakers that talk to you")
                .vector(new float[]{0.1f, 0.15f, 0.3f, 0.12f, 0.05f}),
        new Document("2")
                .append("text", "An AI quilt to help you sleep forever")
                .vector(new float[]{0.45f, 0.09f, 0.01f, 0.2f, 0.11f}),
        new Document("3")
                .append("text", "A deep learning display that controls your mood")
                .vector(new float[]{0.1f, 0.05f, 0.08f, 0.3f, 0.6f}));
    System.out.println("6/7 - Collection populated");

    // ⬇️ NEW CODE

FindIterable<Document> resultsSet = collection.find(
    new float[]{0.15f, 0.1f, 0.1f, 0.35f, 0.55f},
    10
);
resultsSet.forEach(System.out::println);
collection.drop();
System.out.println("Deleted the collection"); 

    // ⬆️ NEW CODE

    }
}

You get a sorted list of the documents you inserted. The database sorts documents by their similarity to the query vector, most similar documents first. The calculation uses cosine similarity by default.

Run the code

Run the code you defined above.

Python
TypeScript
Java

python QuickStartHCD.py

npm

npx tsx QuickStartHCD.ts

Yarn

yarn dlx tsx QuickStartHCD.ts

Maven

mvn clean compile
export OPENAI_API_KEY=<your-api-key>
mvn exec:java -Dexec.mainClass="com.example.QuickStartHCD"

Gradle

gradle build
gradle run

Vector database quickstart with Data API

Objective

Install HCD

Install a terminal to run your client

Identify your credentials

Install the client

Initialize the client

Create a namespace

Create a collection

Load vector embeddings

Use the `$vectorize` method

Use the `$vector` method instead of `$vectorize`

Perform a similarity search

Run the code

See also

Was this helpful?

Give Feedback