Develop applications with Firebase Genkit and Astra DB Serverless

query_builder 30 min

Genkit is an open-source TypeScript toolkit built by the Firebase team that is designed to help you build, test, and deploy AI-powered features in web and mobile apps. You don’t need a Firebase account to use Genkit because Genkit is an open-source framework.

With Genkit, you can leverage the capabilities of LLMs for tasks involving complex data processing, retrieval, and interaction. It offers a unified interface for working with LLMs, monitoring of applications in production, and features that simplify retrieval-augmented generation (RAG) application development. Genkit handles the complexity of AI integration so you can focus on developing user experiences like chatbots, intelligent agents, workflow automation, and recommendation systems.

The Astra DB Genkit plugin provides an Astra DB retriever and indexer for RAG with Genkit.

Prerequisites

To use the Astra DB Genkit plugin, you need the following:

Familiarity with Genkit, including the indexer, retriever, and embedder abstractions for RAG with Genkit
An active Astra account
A Serverless (Vector) database with a vector-enabled collection that contains the data you want your application to use for vector search
An application token with the Database Administrator role for your database
Your database’s Data API endpoint in the form of https://DATABASE_ID-REGION.apps.astra.datastax.com

You can get the endpoint and token in the Astra Portal on your database’s Overview page. For more information, see Generate an application token for a database.

Install and configure the Astra DB Genkit plugin

If you haven’t done so already, install the Genkit framework:
```
npm install genkit
```
For more information about using Genkit, see the Genkit documentation.
Install the Astra DB Genkit plugin:
```
npm install genkitx-astra-db
```
In your project, import the necessary dependencies, and then call genkit() to configure a Genkit instance, passing plugin parameters in the plugins array:
- Use Genkit to generate embeddings
- Use vectorize to generate embeddings
To use the Genkit AI model interface to generate embeddings, you must import an AI model plugin dependency, and then specify the plugin settings, embedder, and embedderOptions in your Genkit instance parameters.

The following example uses the genkit-ai/googleai plugin and the textEmbedding004 model:
// Import Genkit and the Astra DB plugin import { genkit } from "genkit"; import { astraDBIndexerRef, astraDBRetrieverRef, astraDB, } from "genkitx-astra-db"; // Import an AI model plugin import { textEmbedding004, googleAI } from "@genkit-ai/googleai"; // Create a Genkit instance const ai = genkit({ plugins: [ astraDB([ { clientParams: { applicationToken: "ASTRA_DB_APPLICATION_TOKEN", apiEndpoint: "ASTRA_DB_API_ENDPOINT", keyspace: "default_keyspace", }, collectionName: "COLLECTION_NAME", embedder: EMBEDDING_MODEL_NAME, embedderOptions: {}, }, ]), googleAI(), ], });
Replace the following:
ASTRA_DB_APPLICATION_TOKEN and ASTRA_DB_API_ENDPOINT: Provide the corresponding values from the Prerequisites.

In a production scenario, DataStax recommends that you use environment variables or other secure references for these values.

COLLECTION_NAME: The name of the collection that you want to use with Genkit.

If your collection isn’t in the default keyspace, change default_keyspace to the correct keyspace name.

EMBEDDING_MODEL_NAME: Specify the embedding model to use, such as textEmbedding004.

Your collection’s dimension setting must match the embedding model you choose.

embedderOptions: Specify custom embedder options, if any are available for your chosen embedder. Otherwise, omit embedderOptions.

googleAI(): This is the Genkit plugin for Google AI models. To use a different AI model plugin, import the relevant plugin, and then pass the plugin to the plugins array along with any plugin parameters, such as an API key to access the embedding provider’s API.

You must import and configure the AI model plugin that interfaces with your embedder model. For example, you must use the googleAI() plugin to interface with Google’s Gemini models, such as textEmbedding004.
For information about available models, plugins, and their options, see Generating content with AI models and Genkit.
If your collection can automatically generate embeddings with vectorize, Genkit can use your collection’s vectorize integration to generate embeddings.

In this case, you only need the genkit and genkitx-astra-db dependencies. You don’t need to import an AI model plugin, and you don’t need to specify the embedder and embedderOptions parameters in your Genkit instance configuration.
// Import Genkit and the Astra DB plugin import { genkit } from "genkit"; import { astraDBIndexerRef, astraDBRetrieverRef, astraDB, } from "genkitx-astra-db"; // Create a Genkit instance const ai = genkit({ plugins: [ astraDB([ { clientParams: { applicationToken: "ASTRA_DB_APPLICATION_TOKEN", apiEndpoint: "ASTRA_DB_API_ENDPOINT", keyspace: "default_keyspace", }, collectionName: "COLLECTION_NAME", }, ]), ], });
Replace the following:
ASTRA_DB_APPLICATION_TOKEN and ASTRA_DB_API_ENDPOINT: Provide the corresponding values from the Prerequisites.

In a production scenario, DataStax recommends that you use environment variables or other secure references for these values.

COLLECTION_NAME: The name of the collection that you want to use with your Genkit project.

If your collection isn’t in the default keyspace, change default_keyspace to the correct keyspace name.

Use the Astra DB Genkit plugin

The Astra DB Genkit plugin provides an Astra DB retriever and indexer for Genkit.

The following examples show basic usage of the retriever and indexer functions. For more information and detailed usage examples, see Build a RAG Chat App with Firebase Genkit and Astra DB.

Indexer

Get an astraDBIndexer reference using the collectionName and an optional displayName, and then pass the reference to the Genkit function index():

// Get an indexer reference using your collection
export const astraDBIndexer = astraDBIndexerRef({
  collectionName: "COLLECTION_NAME",
  //displayName?: "DISPLAY_NAME",
});

// Pass the required references to the index function
await index({
  indexer: astraDBIndexer,
  documents,
});

Retriever

The retriever performs a vector search on your Astra DB collection based on a given query and options.

Get an astraDBRetriever reference using the collectionName and an optional displayName, and then pass the reference to the Genkit function retrieve():

// Use the collection name to get a reference
export const astraDBRetriever = astraDBRetrieverRef({
  collectionName: "COLLECTION_NAME",
  //displayName?: "DISPLAY_NAME",
});

// Pass the required references to the retrieve function
await retrieve({
  retriever: astraDBRetriever,
  query,
  //options?: {
  //  k: 5,
  //},
});

You can pass options to retrieve() to modify the retriever:

k: The number of documents to return. The default is 5.

filter: A Data API filter to limit the subset of documents queried by the retriever. To enforce type-checking on the filter options, provide a schema type for astraDBRetrieverRef.

The following example calls the retrieve() function with a filter so that the vector search only queries documents where score is greater than 75:

// Define the schema type for three fields in the collection
type Schema = {
  _id: string;
  text: string;
  score: number;
};

// Get a retriever reference using the schema type and collection name
export const astraDBRetriever = astraDBRetrieverRef<Schema>({
  collectionName: "matchResults",
});

// Call the retrieve function with retriever, query, and options
// In options, apply a greater-than filter to the score field
await retrieve({
  retriever: astraDBRetriever,
  query,
  options: {
    filter: {
      score: { $gt: 75 },
    },
  },
});

For all filter operators, see Filter operators for collections.

Build a RAG app with Genkit and Astra DB

For more examples and more detailed usage information, see the tutorial to Build a RAG Chat App with Firebase Genkit and Astra DB. This tutorial demonstrates how to build a Genkit flow that ingests data into your Astra DB collection, and then create a RAG application that answers questions based on that data using Astra DB and other Genkit components.

For a video version of this tutorial, see Video tutorial: Build a RAG App with Firebase Genkit and Astra DB

Report an issue with the Astra DB Genkit plugin

You can report issues with the Astra DB Genkit plugin in the genkitx-astra-db repository.