Hyper-Converged Database (HCD) quickstart for collections
If your data is fully structured and you want to use a fixed schema, see the quickstart for tables instead. |
This quickstart demonstrates how to create a collection, insert data with vector embeddings to the collection, and perform a vector search to find similar data.
To learn more about vector databases and vector search, see About vector databases and What is Vector Search.
Store your endpoint
The Data API endpoint for your database has the form: http://CLUSTER_HOST:GATEWAY_PORT
-
Replace CLUSTER_HOST with the external IP address of any node in your cluster. To find this, run
kubectl get nodes -o wide
and use any of the values listed under "EXTERNAL-IP" in the output. -
Replace GATEWAY_PORT with the port number for your API gateway service. To find this, run
kubectl get svc
and look for the "PORT(S)" value that corresponds toNodePort
.
For this quickstart, store the endpoint in an environment variable:
-
Linux or macOS
-
Windows
export API_ENDPOINT=API_ENDPOINT
set API_ENDPOINT=API_ENDPOINT
Store your username and password
You set a username and password when you create a cluster.
If you didn’t provide superuser credentials when you created your cluster, they were generated automatically and saved in a superuser secret named CLUSTER_NAME-superuser
.
The CLUSTER_NAME-superuser
secret contains both the username and the password.
For this quickstart, store the username and password in environment variables:
-
Linux or macOS
-
Windows
export USERNAME=USERNAME
export PASSWORD=PASSWORD
set USERNAME=USERNAME
set PASSWORD=PASSWORD
Install a client
Install one of the Data API clients to facilitate interactions with the Data API.
-
Python
-
TypeScript
-
Java
-
Update to Python version 3.8 or later if needed.
-
Update to pip version 23.0 or later if needed.
-
Install the latest version of the astrapy package
.
pip install "astrapy>=2.0,<3.0"
-
Update to Node version 18 or later if needed.
-
Update to TypeScript version 5 or later if needed. This is unnecessary if you are using JavaScript instead of TypeScript.
-
Install the latest version of the @datastax/astra-db-ts package
.
For example:
npm install @datastax/astra-db-ts
-
Maven
-
Gradle
-
Update to Java version 17 or later if needed. DataStax recommends Java 21.
-
Update to Maven version 3.9 or later if needed.
-
Add a dependency to the latest version of the astra-db-java package
.
pom.xml<dependencies> <dependency> <groupId>com.datastax.astra</groupId> <artifactId>astra-db-java</artifactId> <version>VERSION</version> </dependency> </dependencies>
-
Update to Java version 17 or later if needed. DataStax recommends Java 21.
-
Update to Gradle version 11 or later if needed.
-
Add a dependency to the latest version of the astra-db-java package
.
build.gradle(.kts)dependencies { implementation 'com.datastax.astra:astra-db-java:VERSION' }
Connect to your database
The following function will connect to your database.
Copy the file into your project. You don’t need to execute the function now; the subsequent code examples will import and use this function.
-
Python
-
TypeScript
-
Java
import os
from astrapy import DataAPIClient, Database
from astrapy.authentication import UsernamePasswordTokenProvider
from astrapy.constants import Environment
def connect_to_database() -> Database:
"""
Connects to your database.
This function retrieves the database endpoint, username, and password from the
environment variables `API_ENDPOINT`, `USERNAME`, and `PASSWORD`.
Returns:
Database: An instance of the connected database.
Raises:
RuntimeError: If the environment variables `API_ENDPOINT`,
`USERNAME`, or `PASSWORD` are not defined.
"""
endpoint = os.environ.get("API_ENDPOINT") (1)
username = os.environ.get("USERNAME")
password = os.environ.get("PASSWORD")
if not endpoint or not username or not password:
raise RuntimeError(
"Environment variables API_ENDPOINT, USERNAME, and PASSWORD must be defined"
)
# Create an instance of the `DataAPIClient` class
client = DataAPIClient(environment=Environment.HCD)
# Get the database specified by your endpoint and provide the token
database = client.get_database(
endpoint,
token=UsernamePasswordTokenProvider(username, password),
)
print(f"Connected to database {database.info().name}")
return database
1 | Store your database’s endpoint, username, and password in environment variables named API_ENDPOINT , USERNAME , and PASSWORD , as instructed in Store your endpoint and Store your username and password. |
import {
DataAPIClient,
Db,
UsernamePasswordTokenProvider,
} from "@datastax/astra-db-ts";
/**
* Connects to your database.
* This function retrieves the database endpoint, username, and password from the
* environment variables `API_ENDPOINT`, `USERNAME`, and `PASSWORD`.
*
* @returns An instance of the connected database.
* @throws Will throw an error if the environment variables
* `API_ENDPOINT`, `USERNAME`, or `PASSWORD` are not defined.
*/
export function connectToDatabase(): Db {
const {
API_ENDPOINT: endpoint,
USERNAME: username,
PASSWORD: password,
} = process.env; (1)
if (!endpoint || !username || !password) {
throw new Error(
"Environment variables API_ENDPOINT, USERNAME, and PASSWORD must be defined.",
);
}
// Create an instance of the `DataAPIClient` class
const client = new DataAPIClient({ environment: "hcd" });
// Get the database specified by your endpoint and provide the token
const database = client.db(endpoint, {
token: new UsernamePasswordTokenProvider(username, password),
});
console.log("Connected to database");
return database;
}
1 | Store your database’s endpoint, username, and password in environment variables named API_ENDPOINT , USERNAME , and PASSWORD , as instructed in Store your endpoint and Store your username and password. |
package com.quickstart;
import com.datastax.astra.client.DataAPIClient;
import com.datastax.astra.client.DataAPIClients;
import com.datastax.astra.client.databases.Database;
public class QuickstartConnect {
/**
* Connects to your database. This function retrieves the database endpoint and
* application token from the environment variables
* `API_ENDPOINT`, `USERNAME`, and `PASSWORD`.
*
* @return an instance of the connected database
* @throws IllegalStateException if the environment variables
* `API_ENDPOINT`, `USERNAME`, or `PASSWORD` are not defined
*/
public static Database connectToDatabase(String keyspace) {
String endpoint = System.getenv("API_ENDPOINT"); (1)
String username = System.getenv("USERNAME");
String password = System.getenv("PASSWORD");
if (endpoint == null || username == null || password == null) {
throw new IllegalStateException(
"Environment variables API_ENDPOINT, USERNAME, and PASSWORD must be defined");
}
// Create an instance of `DataAPIClient`
DataAPIClient client = DataAPIClients.clientHCD(username, password);
// Get the database specified by your endpoint
Database database;
if (keyspace != null && !keyspace.trim().isEmpty()) {
// Connect with the provided keyspace
database = client.getDatabase(endpoint, keyspace);
System.out.println("Connected to database with keyspace: " + keyspace);
} else {
// Default connection without a specific keyspace
database = client.getDatabase(endpoint);
System.out.println("Connected to database (no keyspace specified).");
}
return database;
}
public static Database connectToDatabase() {
return connectToDatabase(null);
}
}
1 | Store your database’s endpoint, username, and password in environment variables named API_ENDPOINT , USERNAME , and PASSWORD , as instructed in Store your endpoint and Store your username and password. |
Create a keyspace
The following script will create a new keyspace in your database.
-
Copy the script into your project.
-
If needed, update the import path to the "connect to database" function from the previous section.
-
Execute the script.
For information about executing scripts, refer to the documentation for the language that your script uses.
Once the script completes, you should see a printed message confirming keyspace creation.
-
Python
-
TypeScript
-
Java
from quickstart_connect import connect_to_database (1)
def main() -> None:
database = connect_to_database()
# Get an admin object
admin = database.get_database_admin()
# Create a keyspace
admin.create_keyspace("quickstart_keyspace") (2)
print(f"Created keyspace")
if __name__ == "__main__":
main()
1 | This is the connect_to_database function from the previous section.
Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
2 | This script creates a keyspace named quickstart_keyspace .
If you want to use a different name, change the name before running the script. |
import { connectToDatabase } from "./quickstart-connect"; (1)
(async function () {
const database = connectToDatabase();
// Get an admin object
const admin = database.admin({ environment: "hcd" });
// Create a keyspace
await admin.createKeyspace("quickstart_keyspace"); (2)
console.log("Created keyspace");
})();
1 | This is the connectToDatabase function from the previous section. Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
2 | This script creates a keyspace named quickstart_keyspace .
If you want to use a different name, change the name before running the script. |
package com.quickstart;
import com.datastax.astra.client.admin.DatabaseAdmin;
import com.datastax.astra.client.databases.Database;
public class QuickstartCreateKeyspaceDemo {
public static void main(String[] args) {
Database database = QuickstartConnect.connectToDatabase(); (1)
// Get an admin object
DatabaseAdmin admin = database.getDatabaseAdmin();
// Create a keyspace
admin.createKeyspace("quickstart_keyspace"); (2)
System.out.println("Created keyspace");
}
}
1 | This is the connectToDatabase function from the previous section.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
2 | This script creates a keyspace named quickstart_keyspace .
If you want to use a different name, change the name before running the script. |
Create a collection
The following script will create an empty collection in your database.
-
Copy the script into your project.
-
If needed, update the import path to the "connect to database" function from the previous section.
-
Execute the script.
For information about executing scripts, refer to the documentation for the language that your script uses.
Once the script completes, you should see a printed message confirming the collection creation.
-
Python
-
TypeScript
-
Java
from quickstart_connect import connect_to_database (1)
from astrapy.constants import VectorMetric
from astrapy.info import (
CollectionDefinition,
CollectionVectorOptions,
)
def main() -> None:
database = connect_to_database()
collection = database.create_collection(
"quickstart_collection", (2)
keyspace="quickstart_keyspace", (3)
definition=CollectionDefinition(
vector=CollectionVectorOptions( (4)
dimension=5,
metric=VectorMetric.COSINE,
),
),
)
print(f"Created collection {collection.full_name}")
if __name__ == "__main__":
main()
1 | This is the connect_to_database function from the previous section.
Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
2 | This script creates a collection named quickstart_collection .
If you want to use a different name, change the name before running the script. |
3 | This script expects that you have a keyspace named quickstart_keyspace .
If you used a different keyspace name in the previous section, update it here. |
4 | This collection will store 5‑dimensional vector data and use the cosine similarity metric for comparisons. |
This example creates an untyped collection, but you can define a client-side type for your collection to help statically catch errors. For examples, see Create a collection and Typing Collections and Tables. |
import { connectToDatabase } from "./quickstart-connect"; (1)
(async function () {
const database = connectToDatabase();
const collection = await database.createCollection(
"quickstart_collection", (2)
{
keyspace: "quickstart_keyspace", (3)
vector: {
(4)
dimension: 5,
metric: "cosine",
},
},
);
console.log(`Created collection ${collection.keyspace}.${collection.name}`);
})();
1 | This is the connectToDatabase function from the previous section. Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
2 | This script creates a collection named quickstart_collection .
If you want to use a different name, change the name before running the script. |
3 | This script expects that you have a keyspace named quickstart_keyspace .
If you used a different keyspace name in the previous section, update it here. |
4 | This collection will store 5‑dimensional vector data and use the cosine similarity metric for comparisons. |
package com.quickstart;
import com.datastax.astra.client.collections.Collection;
import com.datastax.astra.client.collections.definition.CollectionDefinition;
import com.datastax.astra.client.collections.definition.documents.Document;
import com.datastax.astra.client.core.vector.SimilarityMetric;
import com.datastax.astra.client.databases.Database;
public class QuickstartCreateCollectionDemo {
public static void main(String[] args) {
Database database = QuickstartConnect.connectToDatabase("quickstart_keyspace"); (1)
Collection<Document> collection =
database.createCollection(
"quickstart_collection", (2)
new CollectionDefinition() (3)
.vectorDimension(5)
.vectorSimilarity(SimilarityMetric.COSINE));
System.out.println("Created collection " + collection.getCollectionName());
}
}
1 | This is the connectToDatabase function from the previous section.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. If you used a different keyspace name in the previous section, update it here. |
2 | This script creates a collection named quickstart_collection .
If you want to use a different name, change the name before running the script. |
3 | This collection will store 5‑dimensional vector data and use the cosine similarity metric for comparisons. |
Insert data to your collection
The following script will insert data from a JSON file to your collection.
-
Copy the script into your project.
-
Download the quickstart_dataset.json sample dataset (76 kB). This dataset is a JSON array describing library books.
-
Replace
PATH_TO_DATA_FILE
in the script with the path to the dataset. -
If needed, update the import path to the "connect to database" function from the previous section.
-
Execute the script.
For information about executing scripts, refer to the documentation for the language that your script uses.
Once the script completes, you should see a printed message confirming the insertion of 100 documents.
-
Python
-
TypeScript
-
Java
import json
from quickstart_connect import connect_to_database (1)
from astrapy.data_types import DataAPIDate
def main() -> None:
database = connect_to_database()
collection = database.get_collection(
"quickstart_collection", keyspace="quickstart_keyspace"
) (2)
data_file_path = "PATH_TO_DATA_FILE" (3)
# Read the JSON file and parse it into a JSON array
with open(data_file_path, "r", encoding="utf8") as file:
json_data = json.load(file)
# Assemble the documents to insert
documents = [
{
**data,
# Convert the date string into a DataAPIDate
"due_date": (
DataAPIDate.from_string(data["due_date"])
if data.get("due_date")
else None
),
# Populate the reserved $vector field
"$vector": data["summary_genres_vector"],
}
for data in json_data
]
# Insert the data
inserted = collection.insert_many(documents)
print(f"Inserted {len(inserted.inserted_ids)} documents.")
if __name__ == "__main__":
main()
1 | This is the connect_to_database function from the previous section. Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
2 | This script expects that you have a collection named quickstart_collection in a keyspace named quickstart_keyspace .
If you used a different keyspace or collection name in the previous sections, update it here. |
3 | Replace PATH_TO_DATA_FILE with the path to the JSON data file. |
import { connectToDatabase } from "./quickstart-connect"; (1)
import fs from "fs";
(async function () {
const database = connectToDatabase();
const collection = database.collection("quickstart_collection", {
keyspace: "quickstart_keyspace",
}); (2)
const dataFilePath = "PATH_TO_DATA_FILE"; (3)
// Read the JSON file and parse it into a JSON array
const rawData = fs.readFileSync(dataFilePath, "utf8");
const jsonData = JSON.parse(rawData);
// Assemble the documents to insert
const documents = jsonData.map((data: any) => ({
...data,
// Convert the date string into a Date
due_date: data.due_date ? new Date(data.due_date) : null,
// Populate the reserved $vector field
$vector: data.summary_genres_vector,
}));
// Insert the data
const inserted = await collection.insertMany(documents);
console.log(`Inserted ${inserted.insertedCount} documents.`);
})();
1 | This is the connectToDatabase function from the previous section. Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
2 | This script expects that you have a collection named quickstart_collection in a keyspace named quickstart_keyspace .
If you used a different keyspace or collection name in the previous sections, update it here. |
3 | Replace PATH_TO_DATA_FILE with the path to the JSON data file. |
package com.quickstart;
import com.datastax.astra.client.collections.Collection;
import com.datastax.astra.client.collections.commands.results.CollectionInsertManyResult;
import com.datastax.astra.client.collections.definition.documents.Document;
import com.datastax.astra.client.databases.Database;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.File;
import java.io.FileInputStream;
import java.util.List;
public class QuickstartInsertToCollectionDemo {
public static void main(String[] args) throws Exception {
Database database = QuickstartConnect.connectToDatabase("quickstart_keyspace"); (1)
Collection<Document> collection = database.getCollection("quickstart_collection"); (2)
// Read the JSON file
File jsonFile =
new File(
QuickstartInsertToCollectionDemo.class
.getClassLoader()
.getResource("PATH_TO_DATA_FILE") (3)
.toURI());
// Load the JSON file into a list of documents
List<Document> documents =
new ObjectMapper().readValue(new FileInputStream(jsonFile), new TypeReference<>() {});
// Populate the reserved $vector field for each document
List<Document> documentsWithVectorField =
documents.stream()
.peek(
document -> {
if (document.containsKey("summary_genres_vector")) {
document.put("$vector", document.get("summary_genres_vector"));
document.remove("summary_genres_vector");
}
})
.toList();
CollectionInsertManyResult result = collection.insertMany(documentsWithVectorField);
System.out.println("Inserted " + result.getInsertedIds().size() + " documents.");
}
}
1 | This is the connectToDatabase function from the previous section.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. If you used a different keyspace name in the previous sections, update it here. |
2 | This script expects that your keyspace has a collection named quickstart_collection .
If you used a different collection name in the previous sections, update it here. |
3 | Replace PATH_TO_DATA_FILE with the path to the JSON data file. |
Find data in your collection
After you insert data to your collection, you can search the data. In addition to traditional database filtering, you can perform a vector search to find data that is most similar to a search string.
The following script performs three searches on the sample data that you loaded in Insert data to your collection.
-
Python
-
TypeScript
-
Java
from quickstart_connect import connect_to_database (1)
def main() -> None:
database = connect_to_database()
collection = database.get_collection(
"quickstart_collection", keyspace="quickstart_keyspace"
) (2)
# Find documents that match a filter
print("\nFinding books with rating greater than 4.7...")
rating_cursor = collection.find({"rating": {"$gt": 4.7}})
for document in rating_cursor:
print(f"{document['title']} is rated {document['rating']}")
# Perform a vector search to find the closest match to a given vector
print("\nUsing vector search to find a book...")
single_vector_match = collection.find_one(
sort={
"$vector": [0.016326904, -0.031677246, 0.04815674, 0.0033435822, 0.01876831]
}
)
print(f"{single_vector_match['title']} is the best match")
# Combine a filter, vector search, and projection to find the 3 books with
# more than 400 pages that are the closest matches to a given vector,
# and just return the title and author
print(
"\nUsing filters and vector search to find 3 books with more than 400 pages, returning just the title and author..."
)
vector_cursor = collection.find(
{"number_of_pages": {"$gt": 400}},
sort={
"$vector": [0.016326904, -0.031677246, 0.04815674, 0.0033435822, 0.01876831]
},
limit=3,
projection={"title": True, "author": True},
)
for document in vector_cursor:
print(document)
if __name__ == "__main__":
main()
1 | This is the connect_to_database function from the previous section. Update the import path if necessary. |
2 | This script expects that you have a collection named quickstart_collection in a keyspace named quickstart_keyspace .
If you used a different keyspace or collection name in the previous sections, update it here. |
import { connectToDatabase } from "./quickstart-connect"; (1)
(async function () {
const database = connectToDatabase();
const collection = database.collection("quickstart_collection", {
keyspace: "quickstart_keyspace",
}); (2)
// Find documents that match a filter
console.log("\nFinding books with rating greater than 4.7...");
const ratingCursor = collection.find({ rating: { $gt: 4.7 } }, { limit: 10 });
for await (const document of ratingCursor) {
console.log(`${document.title} is rated ${document.rating}`);
}
// Perform a vector search to find the closest match to a given vector
console.log("\nUsing vector search to find a book...");
const singleVectorMatch = await collection.findOne(
{},
{
sort: {
$vector: [
0.016326904, -0.031677246, 0.04815674, 0.0033435822, 0.01876831,
],
},
},
);
console.log(`${singleVectorMatch?.title} is the best match`);
// Combine a filter, vector search, and projection to find the 3 books with
// more than 400 pages that are the closest matches to a search string,
// and just return the title and author
console.log(
"\nUsing filters and vector search to find 3 books with more than 400 pages, returning just the title and author...",
);
const vectorCursor = collection.find(
{ number_of_pages: { $gt: 400 } },
{
sort: {
$vector: [
0.016326904, -0.031677246, 0.04815674, 0.0033435822, 0.01876831,
],
},
limit: 3,
projection: { title: true, author: true },
},
);
for await (const document of vectorCursor) {
console.log(document);
}
})();
1 | This is the connectToDatabase function from the previous section. Update the import path if necessary. |
2 | This script expects that you have a collection named quickstart_collection in a keyspace named quickstart_keyspace .
If you used a different keyspace or collection name in the previous sections, update it here. |
package com.quickstart;
import static com.datastax.astra.client.core.query.Projection.include;
import com.datastax.astra.client.collections.Collection;
import com.datastax.astra.client.collections.commands.options.CollectionFindOneOptions;
import com.datastax.astra.client.collections.commands.options.CollectionFindOptions;
import com.datastax.astra.client.collections.definition.documents.Document;
import com.datastax.astra.client.core.query.Filter;
import com.datastax.astra.client.core.query.Filters;
import com.datastax.astra.client.core.query.Sort;
import com.datastax.astra.client.databases.Database;
public class QuickstartFindDemo {
public static void main(String[] args) {
Database database = QuickstartConnect.connectToDatabase("quickstart_keyspace"); (1)
Collection<Document> collection = database.getCollection("quickstart_collection"); (2)
// Find documents that match a filter
System.out.println("\nFinding books with rating greater than 4.7...");
Filter filter = Filters.gt("rating", 4.7);
CollectionFindOptions options = new CollectionFindOptions().limit(10);
collection
.find(filter, options)
.forEach(
document -> {
System.out.println(
document.getString("title") + " is rated " + document.get("rating"));
});
// Perform a vector search to find the closest match to a given vector
System.out.println("\nUsing vector search to find a book...");
CollectionFindOneOptions options2 =
new CollectionFindOneOptions()
.sort(
Sort.vector(
new float[] {
0.016326904f, -0.031677246f, 0.04815674f, 0.0033435822f, 0.01876831f
}));
collection
.findOne(options2)
.ifPresent(
document -> {
System.out.println(document.getString("title") + " is the best match");
});
// Combine a filter, vector search, and projection to find the 3 books with
// more than 400 pages that are the closest matches to a given vector,
// and just return the title and author
System.out.println(
"\nUsing filters and vector search to find 3 books with more than 400 pages, returning just the title and author...");
Filter filter3 = Filters.gt("number_of_pages", 400);
CollectionFindOptions options3 =
new CollectionFindOptions()
.limit(3)
.sort(
Sort.vector(
new float[] {
0.016326904f, -0.031677246f, 0.04815674f, 0.0033435822f, 0.01876831f
}))
.projection(include("title", "author"));
collection
.find(filter3, options3)
.forEach(
document -> {
System.out.println(document);
});
}
}
1 | This is the connectToDatabase function from the previous section.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. If you used a different keyspace name in the previous sections, update it here. |
2 | This script expects that your keyspace has a collection named quickstart_collection .
If you used a different collection name in the previous sections, update it here. |
Next steps
For more practice, you can continue building with the collection that you created here. For example, try inserting more data to the collection, or try different searches. The Data API reference provides code examples for various operations.
Upload data from different sources
This quickstart demonstrated how to insert data from a JSON file, but you can insert data from many sources, including CSV and PDF files.
This quickstart also demonstrated how to insert data to a collection, which uses a flexible schema. If your data is structured and you want to use a fixed schema, you can use a table instead of a collection. See the quickstart for tables.
Perform more complex searches
This quickstart demonstrated how to find data using filters and vector search. To learn more about the searches you can perform, see Find a document, Find documents, and Find data with vector search.