Hyper-Converged Database (HCD) quickstart for collections (Go)
|
If your data is fully structured and you want to use a fixed schema, see the quickstart for tables instead. |
This quickstart demonstrates how to create a collection, insert data with vector embeddings to the collection, and perform a vector search to find similar data.
To learn more about vector databases and vector search, see About vector databases and What is Vector Search.
Store your endpoint
The Data API endpoint for your database has the form: http://CLUSTER_HOST:GATEWAY_PORT
-
Replace CLUSTER_HOST with the external IP address of any node in your cluster. To find this, run
kubectl get nodes -o wideand use any of the values listed under "EXTERNAL-IP" in the output. -
Replace GATEWAY_PORT with the port number for your API gateway service. To find this, run
kubectl get svcand look for the "PORT(S)" value that corresponds toNodePort.
For this quickstart, store the endpoint in an environment variable:
-
Linux or macOS
-
Windows
export API_ENDPOINT=API_ENDPOINT
set API_ENDPOINT=API_ENDPOINT
Store your username and password
You set a username and password when you create a cluster.
If you didn’t provide superuser credentials when you created your cluster, they were generated automatically and saved in a superuser secret named CLUSTER_NAME-superuser.
The CLUSTER_NAME-superuser secret contains both the username and the password.
For this quickstart, store the username and password in environment variables:
-
Linux or macOS
-
Windows
export USERNAME=USERNAME
export PASSWORD=PASSWORD
set USERNAME=USERNAME
set PASSWORD=PASSWORD
Install a client
Install one of the Data API clients to facilitate interactions with the Data API.
-
Update to Go version 1.23 or later if needed.
-
Install the latest version of the astra-db-go package.
For example:
go get github.com/datastax/astra-db-go/v2
Connect to your database
The following function will connect to your database.
Copy the file into your project. You don’t need to execute the function now; the subsequent code examples will import and use this function.
package shared
import (
"fmt"
"log"
"os"
"github.com/datastax/astra-db-go/v2/astra"
"github.com/datastax/astra-db-go/v2/astra/options"
)
// ConnectToDatabase connects to a database.
// This function retrieves the database endpoint, username, and password
// from the environment variables `API_ENDPOINT`, `USERNAME`, and
// `PASSWORD`.
//
// Returns an instance of the connected database.
// Exits with an error if the environment variables
// `API_ENDPOINT`, `USERNAME`, or `PASSWORD` are not defined.
func ConnectToDatabase() *astra.Db {
endpoint := os.Getenv("API_ENDPOINT") (1)
username := os.Getenv("USERNAME")
password := os.Getenv("PASSWORD")
if endpoint == "" || username == "" || password == "" {
log.Fatal(
"Environment variables API_ENDPOINT, USERNAME, and PASSWORD must be defined.",
)
}
// Create an instance of DataAPIClient
client := astra.NewClient(
options.API().SetEnvironment(options.EnvironmentHCD),
)
// Get the database specified by your endpoint and provide the token
database := client.Database(
endpoint,
options.API().
SetUsernamePasswordTokenProvider(
"**USERNAME**",
"**PASSWORD**",
),
)
fmt.Printf("Connected to database %s\n", database.Endpoint())
return database
}
| 1 | Store your database’s endpoint, username, and password in environment variables named API_ENDPOINT, USERNAME, and PASSWORD, as instructed in Store your endpoint and Store your username and password. |
Create a keyspace
The following code will create a new keyspace in your database.
-
Copy the code into your project.
-
If needed, update the import path to the "connect to database" function from the previous section.
-
Execute the code.
For information about executing code, refer to the documentation for your programming language.
Once the code completes, you should see a printed message confirming keyspace creation.
package main
import (
"context"
"fmt"
"log"
"quickstart/shared"
)
func main() {
ctx := context.Background()
database := shared.ConnectToDatabase() (1)
// Get an admin object
admin, err := database.DatabaseAdmin()
if err != nil {
log.Fatal(err)
}
// Create a keyspace
err = admin.CreateKeyspace(ctx, "quickstart_keyspace") (2)
if err != nil {
log.Fatal(err)
}
fmt.Println("Created keyspace")
}
| 1 | This is the connectToDatabase function from the previous section.
Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
| 2 | This code creates a keyspace named quickstart_keyspace.
If you want to use a different name, change the name before running the code. |
Create a collection
The following code will create an empty collection in your database.
-
Copy the code into your project.
-
If needed, update the import path to the "connect to database" function from the previous section.
-
Execute the code.
For information about executing code, refer to the documentation for your programming language.
Once the code completes, you should see a printed message confirming the collection creation.
package main
import (
"context"
"fmt"
"log"
"quickstart/shared"
"github.com/datastax/astra-db-go/v2/astra/options"
)
func main() {
ctx := context.Background()
database := shared.ConnectToDatabase() (1)
collection, err := database.CreateCollection(
ctx,
"quickstart_collection", (2)
options.CreateCollection().
SetKeyspace("quickstart_keyspace"). (3)
UpdateVector(
options.Vector(). (4)
SetDimension(5).
SetMetric(options.MetricCosine),
),
)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Created collection %s\n", collection.Name())
}
| 1 | This is the connectToDatabase function from the previous section.
Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
| 2 | This code creates a collection named quickstart_collection.
If you want to use a different name, change the name before running the code. |
| 3 | This code expects that you have a keyspace named quickstart_keyspace.
If you used a different keyspace name in the previous section, update it here. |
| 4 | This collection will store 5-dimensional vector data and use the cosine similarity metric for comparisons. |
Insert data to your collection
The following code will insert data from a JSON file to your collection.
-
Copy the code into your project.
-
Download the quickstart_dataset.json sample dataset (76 kB). This dataset is a JSON array describing library books.
-
Replace
PATH_TO_DATA_FILEin the code with the path to the dataset. -
If needed, update the import path to the "connect to database" function from the previous section.
-
Execute the code.
For information about executing code, refer to the documentation for your programming language.
Once the code completes, you should see a printed message confirming the insertion of 100 documents.
package main
import (
"context"
"encoding/json"
"fmt"
"log"
"os"
"time"
"quickstart/shared"
"github.com/datastax/astra-db-go/v2/astra"
"github.com/datastax/astra-db-go/v2/astra/options"
)
func main() {
ctx := context.Background()
database := shared.ConnectToDatabase() (1)
collection := database.Collection(
"quickstart_collection",
options.GetCollection().SetKeyspace("quickstart_keyspace"),
) (2)
dataFilePath := "PATH_TO_DATA_FILE" (3)
// Read the JSON file and parse it into a JSON array
rawData, err := os.ReadFile(dataFilePath)
if err != nil {
log.Fatal(err)
}
var jsonData []map[string]any
if err := json.Unmarshal(rawData, &jsonData); err != nil {
log.Fatal(err)
}
// Assemble the documents to insert:
// - Convert the date string into a time.Time
// - Add a $vector field
documents := make([]astra.Document, len(jsonData))
for i, data := range jsonData {
// Convert due_date string to time.Time if present
if dueDateStr, ok := data["due_date"].(string); ok &&
dueDateStr != "" {
dueDate, err := time.Parse("2006-01-02", dueDateStr)
if err != nil {
log.Fatalf("invalid due_date %q: %v", dueDateStr, err)
}
data["due_date"] = dueDate
}
// Populate the reserved $vector field
if vector, ok := data["summary_genres_vector"]; ok {
data["$vector"] = vector
} else {
log.Fatalf("record %d missing summary_genres_vector", i)
}
documents[i] = astra.NewDocument(data)
}
// Insert the data
insertedResult, err := collection.InsertMany(
ctx,
documents,
)
if err != nil {
log.Fatal(err)
}
fmt.Printf(
"Inserted %d documents.\n",
insertedResult.InsertedCount(),
)
}
| 1 | This is the connectToDatabase function from the previous section.
Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
| 2 | This code expects that you have a collection named quickstart_collection in a keyspace named quickstart_keyspace.
If you used a different keyspace or collection name in the previous sections, update it here. |
| 3 | Replace PATH_TO_DATA_FILE with the path to the JSON data file. |
Find data in your collection
After you insert data to your collection, you can search the data. In addition to traditional database filtering, you can perform a vector search to find data that is most similar to a search vector.
The following code performs three searches on the sample data that you loaded in Insert data to your collection.
package main
import (
"context"
"fmt"
"log"
"quickstart/shared"
"github.com/datastax/astra-db-go/v2/astra"
"github.com/datastax/astra-db-go/v2/astra/filter"
"github.com/datastax/astra-db-go/v2/astra/options"
"github.com/datastax/astra-db-go/v2/astra/sort"
)
func main() {
ctx := context.Background()
database := shared.ConnectToDatabase() (1)
collection := database.Collection(
"quickstart_collection",
options.GetCollection().SetKeyspace("quickstart_keyspace"),
) (2)
// Find documents that match a filter
fmt.Println("\nFinding books with rating greater than 4.7...")
ratingCursor := collection.Find(
filter.Gt("rating", 4.7),
options.CollectionFind().SetLimit(10),
)
defer ratingCursor.Close()
for ratingCursor.Next(ctx) {
var document astra.Document
if err := ratingCursor.Decode(&document); err != nil {
log.Fatal(err)
}
fmt.Printf(
"%v is rated %v\n",
document.MustGet("title"),
document.MustGet("rating"),
)
}
if err := ratingCursor.Err(); err != nil {
log.Fatal(err)
}
// Perform a vector search to find the closest match to a given vector
fmt.Println("\nUsing vector search to find a book...")
var singleVectorMatch astra.Document
err := collection.FindOne(
ctx,
nil,
options.CollectionFindOne().
SetSort(sort.Vector([]float32{0.016326904, -0.031677246, 0.04815674, 0.0033435822, 0.01876831})),
).Decode(&singleVectorMatch)
if err != nil {
log.Fatal(err)
}
fmt.Printf(
"%v is the best match\n",
singleVectorMatch.MustGet("title"),
)
// Combine a filter, vector search, and projection to find the 3 books
// with
// more than 400 pages that are the closest matches to a search vector,
// and just return the title and author
fmt.Println(
"\nUsing filters and vector search to find 3 books with more than 400 pages, returning just the title and author...",
)
vectorCursor := collection.Find(
filter.Gt("number_of_pages", 400),
options.CollectionFind().
SetSort(sort.Vector([]float32{0.016326904, -0.031677246, 0.04815674, 0.0033435822, 0.01876831})).
SetLimit(3).
SetProjection(map[string]any{"title": true, "author": true}),
)
defer vectorCursor.Close()
for vectorCursor.Next(ctx) {
var document astra.Document
if err := vectorCursor.Decode(&document); err != nil {
log.Fatal(err)
}
fmt.Println(document.ToMap())
}
if err := vectorCursor.Err(); err != nil {
log.Fatal(err)
}
}
| 1 | This is the connectToDatabase function from the previous section.
Update the import path if necessary.
To use the function, ensure you stored your database’s endpoint, username, and password in environment variables as instructed in Store your endpoint and Store your username and password. |
| 2 | This code expects that you have a collection named quickstart_collection in a keyspace named quickstart_keyspace.
If you used a different keyspace or collection name in the previous sections, update it here. |
Next steps
For more practice, you can continue building with the collection that you created here. For example, try inserting more data to the collection, or try different searches. The Data API reference provides code examples for various operations.
- Insert data from different sources
-
This quickstart demonstrated how to insert data from a JSON file, but you can insert data from many sources, including CSV and PDF files.
This quickstart also demonstrated how to insert data to a collection, which uses a flexible schema. If your data is structured and you want to use a fixed schema, you can use a table instead of a collection. See the quickstart for tables.
- Perform more complex searches
-
This quickstart demonstrated how to find data using filters and vector search. To learn more about the searches you can perform, see Find a document (Go), Find documents (Go), and Find data with vector search.