Vector Store RAG
Build a Retrieval Augmented Generation (RAG) application with Astra DB and Langflow.
RAG applications level-up your chatbots with more specialized context loaded and retrieved from your own data.
This starter template contains two flows that must be run in order:
-
The Load Data flow loads data from a local file into the Astra DB vector store.
-
The Retriever flow generates responses to your queries from the documents stored in Astra DB.
This starter project highlights the use of Astra DB in a vector RAG application, but the Astra DB component is adaptable as a vector database for any Langflow project. For more information, see Create an Astra DB component in Langflow.
Prerequisites
-
An active Serverless (Vector) database
Open Langflow and start a new project
-
In the Astra Portal header, switch your active app from Astra DB Serverless to Langflow.
-
Click New Flow, and then select the Vector Store RAG project.
This opens a starter project with the necessary components to run a RAG application using Astra DB.
Run the load data flow
The load data flow ingests documents into the Astra DB database, and it has the following components:
-
A File component that uploads a text file to Langflow.
-
A Split Text component that splits the text into smaller chunks.
-
An OpenAI embeddings component that generates embeddings for the text chunks.
-
An Astra DB component that stores the embeddings in the Astra DB database.
To create the load data flow:
-
Add your credentials to the OpenAI components. The fastest way to complete these fields is with Langflow’s Global Variables.
-
In the OpenAI API Key field, click the Globe icon, and then click Add New Variable. Alternatively, click your username in the top right corner and select Settings.
-
-
Click Global Variables, and then click Add New.
-
Name your variable. Paste your OpenAI API key (
sk-…) in the Value field. -
In the Apply To Fields field, select the OpenAI API Key field to apply this variable to all OpenAI Embeddings components.
-
-
Add your credentials to the AstraDB component using the same Global Variables process.
-
In the Astra DB Application Token field, click the and then click Add New Variable, or click your username in the top right corner and select Settings.
-
Name your variable. Paste your Astra token (
AstraCS:…) in the Value field. -
In the Apply To Fields field, select the Astra DB Application Token field to apply this variable to all Astra components.
-
-
Configure the Astra DB component.
-
In the Astra DB Application Token field, add your Astra DB application token. If you’ve created a global variable, select the Astra DB application token from the dropdown list. The component connects to your database and populates the menus with existing databases and collections.
-
Select your Database. If you don’t have a database, select New database. Complete the Name, Cloud provider, and Region fields, and then click Create. Database creation takes a few minutes.
-
Select your Collection. Collections are created in your Astra DB Serverless deployment for storing vector data.
If you select a collection embedded with NVIDIA through Astra’s vectorize service, the Embedding Model port is removed, because you have already generated embeddings for this collection with the NVIDIA
NV-Embed-QAmodel. The component fetches the data from the collection, and uses the same embeddings for queries.
-
-
If you don’t have a collection, create a new one within the component.
-
Select New collection.
-
Complete the Name, Embedding generation method, Embedding model, and Dimensions fields, and then click Create. Your choice for the Embedding generation method and Embedding model depends on whether you want to use embeddings generated by a provider through Astra’s vectorize service, or generated by a component in Langflow.
-
To use embeddings generated by a provider through Astra’s vectorize service, select the model from the Embedding generation method dropdown menu, and then select the model from the Embedding model dropdown menu.
-
To use embeddings generated by a component in Langflow, select Bring your own for both the Embedding generation method and Embedding model fields. In this starter project, the option for the embeddings method and model is the OpenAI Embeddings component connected to the Astra DB component.
-
The Dimensions value must match the dimensions of your collection. This field is not required if you use embeddings generated through Astra’s vectorize service. You can find this value in the Collection in your Astra DB deployment. For more information, see the DataStax Astra DB Serverless documentation.
-
-
-
In the File component, upload a text file from your local machine with data you want to ingest into the Astra DB database.
-
In the Astra DB component, click Play to start the load data flow. Your file passes through the Split Text component, which splits the text into smaller chunks. These chunks become units of meaning when the OpenAI Embeddings component embeds them as vectors into the database.
Run the retriever flow
The retriever flow generates responses to your queries from the embedded documents, and it has the following components:
-
The Chat Input component accepts user input to the chat.
-
An OpenAI embeddings component that generates embeddings from the user input, which is compared to the vector data in the database.
-
An Astra DB component that stores the embeddings in the Astra DB database.
-
The Data to message component parses and converts data into plain text. Prior to Langflow version 1.1.3, this component was named Parse Data.
-
The Prompt component combines the user input with a user-defined prompt.
-
The OpenAI model component sends the user input and prompt to the OpenAI API and receives a response.
-
The Chat Output component prints the flow’s output to the chat.
If you used Langflow’s Global Variables feature, the RAG application flow components are already configured with the necessary credentials.
To run the retriever flow:
-
Click the Playground to start a chat session. Because this flow includes Chat Input and Chat Output components, the Playground displays a Chat Input field and an Extracted Chunks output section.
-
Enter a query, and then make sure the bot responds based on your uploaded data.
