JSON API QuickStart with Mongoose
Time to complete: 10 minutes
For JavaScript developers, the fastest way to create AI applications with vector search capabilities is to use the new Astra DB JSON API.
-
Use Astra Portal to create an Astra DB Vector database in minutes.
-
Generate ID settings associated with your vector database, including an auth key.
-
Then after making a simple configuration change in your existing Node.js apps that use the popular Mongoose library, you can connect to the Astra DB Vector database. Your apps can continue to perform Mongoose operations as usual. In most cases, no other updates are necessary in your Node.js Mongoose code. The Mongoose configuration change involves the
setDriver
function and addingisAstra: true
to theuri
parameter ofmongoose.connect
, as shown in the Code-level details section below.
This goal of this QuickStart is to walk you through the "movies" sample app introduced on the Connect tab of Astra Portal, and to provide extra details about the environment and Node.js files that comprise the app. The source files that comprise the movies app, also known as the |
To participate in this JSON API public preview release, sign into Astra Portal and create an Astra DB Vector database. Follow the instructions on the Connect tab and in this JSON API documentation. At this time, the JSON API is for development environments only. Do not use this API in production. For more, see the DataStax Preview Terms. |
For details about using the JSON API calls directly via curl
commands, in the Swagger UI’s Try It, or in Postman, see Developing with JSON API.
Prerequisites
If you haven’t already:
-
Get an OpenAI API key from https://platform.openai.com/
-
Create an Astra DB Vector database in Astra Portal
-
Install Node.js
QuickStart steps
-
This first step is optional. If you decide to use an OpenAI API key with this demo app, and haven’t yet generated a key, sign into your openai.com account. Open the drop-down menu under your name, and select View API keys. Click Create new secret key. Copy the displayed value, and be sure to keep it secret, keep it safe.
-
Sign into Astra Portal.
-
Create an Astra DB Vector database by clicking the option under Quick Access. On the displayed form, enable the Vector option and enter a database name, keyspace name, your preferred cloud provider, and an available region.
-
Click Create Database.
-
Wait a few minutes for your vector database to reach an Active status.
-
On the Connect tab, click Generate Token for a Database Administrator role. Copy or download the generated ID values.
-
In Select a Method, notice that the JSON API (Preview) card is active.
-
In Connect with the JSON API, click Generate Config to generate a sample application configuration.
-
Download the
astradb-mongoose-config.json
file to your Downloads folder. The JSON file contains the appropriate ID values for your:-
ASTRA_DB_ID
-
ASTRA_DB_REGION
-
ASTRA_DB_KEYSPACE
-
ASTRA_DB_APPLICATION_TOKEN
These variables will allow you to connect your Terminal session to your Astra DB Vector database. A script that you’re about to run will use the defined ID values.
-
-
In a Terminal, run:
npx create-astradb-mongoose-app@latest
The
npx
script displays three prompts:-
The expected location of the
astradb-mongoose-config.json
file that you downloaded from Astra Portal in the previous step. Example:✔ Where is your downloaded Astra DB configuration file located? … /Users/my.name/Downloads/astradb-mongoose-config.json
-
An optional prompt to use vector search in the app. If you enter Yes, you’ll need to provide your OpenAI API key:
✔ Do you want to enable vector search functionality (you will need a funded OpenAI account)? … No / Yes
-
The optional prompt for your OpenAI API key, if you entered Yes:
✔ Awesome! What is your OpenAI API key? … ***************************************************
-
-
The
npx
script displays confirmations:added 73 packages, and audited 74 packages in 5s 7 packages are looking for funding run `npm fund` for details found 0 vulnerabilities 🎉 Congrats! You have successfully created a new Astra DB application with Mongoose! 👉 Next steps: 1. Go to the newly created project folder. cd astradb-mongoose-app 2. Run the sample code. npm start 3. Enjoy development! 😍
-
Go to the
astradb-mongoose-app
folder:cd astradb-mongoose-app
-
Use npm (previously installed with Node) to run the app:
npm start
-
The app loads a dataset to your connected Astra DB Vector database with a few categorized movie summaries. The app can find a movie based on your favorite genre. Example where Western is selected from the list of movies:
1️⃣ With the data loaded, I can find a movie based on your favorite genre. ? What kind of movie would you like to watch? › - Use arrow-keys. Return to submit. Comedy Drama ❯ Western Romance ✔ What kind of movie would you like to watch? › Western Sure! Here is an option for you: The Girl of the Golden West (Western, 1915) The Girl of the Golden West is a surviving 1915 American Western silent black-and-white film directed by Cecil B. DeMille. It was based on the 1905 play The Girl of the Golden West by David Belasco. Prints of the film survive in the Library of Congress film archive. It was the first of four film adaptations that have been made of the play.
-
You can then use a vector search by entering a general query; that is, what you want to watch. Example:
2️⃣ You can also simply describe what you are looking for, and I will find relevant movies. I will use vector search! ? Just tell me what you want to watch... › Something funny
The app performs a vector search and returns three relevant results, such as:
Laughing Gas (Comedy, 1907) Laughing Gas is the title of several short American movies whose plot revolves around real or would-be dentists. -- Bob's Baby (Comedy, 1913) Bob's Baby is a 1913 American comedy film. -- The Sanitarium (Comedy, 1910) The Sanitarium is a 1910 short comedy film featuring Fatty Arbuckle.
-
You can then combine a genre with a general search:
3️⃣ Finally, let's combine the two... ✔ First, what genre are you interested in? › Drama ✔ And now describe to me what you are looking for... Detectives and criminals Here are the two most relevant movies based on your request: The Criminal Hypnotist (Drama, 1909) The Criminal Hypnotist is a 1909 American silent short film directed by D. W. Griffith. -- The Honor of Thieves (Drama, 1909) The Honor of Thieves is a 1909 American short silent drama film directed by D. W. Griffith.
Code-level details
Now let’s look at the files that comprises this Node.js movies app. In an editor such as Visual Code Studio, navigate to the astradb-mongoose-app
folder you’ve used in this QuickStart.
The source files are also available in this GitHub repo. |
.env
Open the .env
file that set up your Terminal session’s connection to your Astra DB Vector database. Notice the defined properties are:
-
ASTRA_DB_ID
-
ASTRA_DB_REGION
-
ASTRA_DB_KEYSPACE
-
ASTRA_DB_APPLICATION_TOKEN
-
Plus your
OPENAI_API_KEY
, if you (optionally) entered its value to then run the app with vector search functionality.
astradb-mongoose.js
The Node.js code includes astradb-mongoose.js
.
Notice how |
This astradb-mongoose.js
module provides a function for establishing a connection to an Astra DB database using Mongoose. Key features:
-
Import Dependencies: The module imports mongoose for database operations and
{ driver, createAstraUri }
from stargate-mongoose to handle the connection with your Astra DB. -
Environment Variables: The
connectToAstraDb
function reads environment variables (process.env
) to configure the database connection. Again, the variables are:-
ASTRA_DB_ID
: ID of the Astra DB instance. -
ASTRA_DB_REGION
: Geographical region where the database is hosted. -
ASTRA_DB_KEYSPACE
: The keyspace to connect to. -
ASTRA_DB_APPLICATION_TOKEN
: Authentication token for the application.
-
-
Construct URI: Uses
createAstraUri
to construct the connection URI dynamically using the provided environment variables. -
Configure Mongoose:
-
mongoose.set("autoCreate", true)
enables auto-creation of the database schema. -
mongoose.setDriver(driver)
sets a custom database driver provided bystargate-mongoose
.
-
-
Connect to Astra DB: Establishes an asynchronous connection to Astra DB using
mongoose.connect()
with the constructed URI and an option flagisAstra: true
. -
Export: Finally, the function
connectToAstraDb
is exported as a module to be used in other parts of the application.
In summary, this module acts as a reusable utility for connecting to Astra DB, abstracting away the details and allowing other parts of your application to connect to the database by simply invoking connectToAstraDb()
.
Here’s the astradb-mongoose.js
source:
const mongoose = require("mongoose");
const { driver, createAstraUri } = require("stargate-mongoose");
const connectToAstraDb = async () => {
const uri = createAstraUri(
process.env.ASTRA_DB_ID,
process.env.ASTRA_DB_REGION,
process.env.ASTRA_DB_KEYSPACE,
process.env.ASTRA_DB_APPLICATION_TOKEN,
);
mongoose.set("autoCreate", true);
mongoose.setDriver(driver);
await mongoose.connect(uri, {
isAstra: true,
});
};
module.exports = { connectToAstraDb };
app.js
Finally, let’s look at app.js
, which is the Node.js app with mongoose operations to perform the actual queries.
The app.js file is entirely a Mongoose implementation: create the model, load the data, and submit queries. There is nothing specific to Astra DB, or that needs to be changed for Astra DB. |
This Node.js app implements the following key features:
-
Environment Variable Configuration: Using the
dotenv
package, the application reads environment variables from a.env
file. -
Mongoose and Astra DB Setup: The script uses Mongoose to define a schema for
movies
and connect to Astra DB, facilitated by the importedconnectToAstraDb
function. -
Data Loading: The
loadData
function:-
Drops the existing
movies
collection in the database, if it exists. -
Defines a new schema for movies, including a special
$vector
field that is used for vector-based searches. -
Inserts movies from a local JSON file into the database in batches of 20.
-
-
User Interaction: Uses the
prompts
library to get user input, and thechalk
library to colorize output. It has three main search features:-
findMovieByGenre
: Asks the user to select a genre and then finds a movie in that genre. -
findMovieByDescription
: Asks the user for a description, then utilizes agenerateEmbedding
function (leveraging OpenAI’s API in this case) to perform a semantic vector search. -
findMovieByGenreAndDescription
: Combines both the genre and the description-based searches.
-
-
Search Sorting and Limiting: The application uses the
$vector
field in the database for semantic matching and ranking of search results. -
OpenAI API Integration: The script has optional OpenAI API functionality for semantic searches. If the API key is available, it enables the description-based and combined searches.
-
Error Handling: Catches and displays errors, highlighting them in red text using
chalk
. -
Asynchronous Execution: All database operations and user interactions are performed asynchronously using
async/await
.
Here’s the app.js
source:
require("dotenv").config();
const prompts = require("prompts");
const chalk = require("chalk");
const mongoose = require("mongoose");
const { connectToAstraDb } = require("./astradb-mongoose");
const { generateEmbedding, movieToString, moviesToString } = require("./util");
const loadData = async () => {
await mongoose.connection.dropCollection("movies");
const Movie = mongoose.model(
"Movie",
new mongoose.Schema(
{
title: String,
year: Number,
genre: String,
description: String,
$vector: {
type: [Number],
validate: (vector) => vector && vector.length === 1536,
},
},
{
collectionOptions: {
vector: {
size: 1536,
function: "cosine",
},
},
},
),
);
const movies = require("./movies.json");
for (let i = 0; i < movies.length; i += 20) {
await Movie.insertMany(movies.slice(i, i + 20));
}
};
const findMovieByGenre = async () => {
const { genre } = await prompts({
type: "select",
name: "genre",
message: "What kind of movie would you like to watch?",
choices: [
{ title: "Comedy", value: "Comedy" },
{ title: "Drama", value: "Drama" },
{ title: "Western", value: "Western" },
{ title: "Romance", value: "Romance" },
],
});
const movie = await mongoose.model("Movie").findOne({ genre });
console.log(`Sure! Here is an option for you:
${movieToString(movie)}
`);
};
const findMovieByDescription = async () => {
const { prompt } = await prompts({
type: "text",
name: "prompt",
message: "Just tell me what you want to watch...",
});
const embedding = await generateEmbedding(prompt);
const movies = await mongoose
.model("Movie")
.find({})
.sort({ $vector: { $meta: embedding } })
.limit(3);
console.log(`Here are three most relevant movies based on your request:
${moviesToString(movies)}
`);
};
const findMovieByGenreAndDescription = async () => {
const { genre, prompt } = await prompts([
{
type: "select",
name: "genre",
message: "First, what genre are you interested in?",
choices: [
{ title: "Comedy", value: "Comedy" },
{ title: "Drama", value: "Drama" },
{ title: "Western", value: "Western" },
{ title: "Romance", value: "Romance" },
],
},
{
type: "text",
name: "prompt",
message: "And now describe to me what you are looking for...",
},
]);
const embedding = await generateEmbedding(prompt);
const movies = await mongoose
.model("Movie")
.find({ genre })
.sort({ $vector: { $meta: embedding } })
.limit(2);
console.log(`Here are two most relevant movies based on your request:
${moviesToString(movies)}
`);
};
(async () => {
try {
await connectToAstraDb();
process.stdout.write("0️⃣ Loading the data to Astra DB... ");
await loadData();
process.stdout.write(" DONE\n\n");
console.log(
"1️⃣ With the data loaded, I can find a movie based on your favorite genre.",
);
await findMovieByGenre();
if (process.env.OPENAI_API_KEY) {
console.log(
"2️⃣ You can also simply describe what you are looking for, and I will find relevant movies. I will use vector search!",
);
await findMovieByDescription();
console.log("3️⃣ Finally, let's combine the two...");
await findMovieByGenreAndDescription();
} else {
console.log(
`🚫 I can't generate embeddings without an OpenAI API key.
Please set the ${chalk.bold(
"OPENAI_API_KEY",
)} environment variable or review the code to see how you can use vector search.`,
);
}
} catch (e) {
console.error(chalk.bold.red("[ERROR] ") + e);
}
})();
Summary
In summary, astradb-mongoose.js
is responsible for setting up a database connection to Astra DB, while app.js
contains the application logic for loading movie data into the database and allowing the user to find a movie by genre and vector search.
What’s next?
For details, see Developing with JSON API.