Build a chatbot with LangChain

query_builder 25 min

The DataStax Astra DB Serverless (Vector) documentation site is currently in Public Preview and is provided on an “AS IS” basis, without warranty or indemnity of any kind. For more, see the DataStax Preview Terms.

Objective

In this tutorial, you will build a chatbot that can communicate with the FAQs from DataStax’s website. You will generate embeddings with LangChain and OpenAI, which the chatbot will use to find relevant content.

You’ll also create a simple React.js chatbot frontend that works with FastAPI and Vercel’s serverless functions to connect your frontend to AstraDB and the OpenAI Completions API.

Architecture Diagram

Prerequisites

To complete this tutorial, you’ll need the following:

You should also be proficient in the following tasks:

  • Interacting with databases.

  • Running a basic Python script.

  • Entering parameters in a user interface for Vercel.

Create your vector database

  1. In the Astra Portal, select Databases in the main navigation.

  2. Click Create Database.

  3. In the Create Database dialog, select the Serverless (Vector) deployment type.

  4. In the Configuration section, enter a name for the new database in the Database name field.

    Since database names can’t be changed later, it’s best to name your database something meaningful. Database names must start and end with an alphanumeric character, and may contain only the following special characters: & + - _ ( ) < > . , @.

  5. Select your preferred Provider and Region.

    You can select from a limited number of regions if you’re on the Free plan. Regions with a lock icon require that you upgrade to a Pay As You Go plan.

    Not all regions may be available. If you don’t see your preferred region listed, please submit a support ticket or send us a message using our live chat in the bottom right of the Astra Portal.

  6. Click Create Database.

    You are redirected to your new database’s Overview screen. Your database starts in Pending status before transitioning to Initializing. You’ll receive a notification once your database is initialized.

Set your environment variables

  1. Under the database name, find the fingerprint icon, and then click the clipboard icon to copy the database ID to use for your ASTRA_DB_ID environment variable.

    Copy Database ID 
    export ASTRA_DB_ID=<database ID>

  2. On the Overview tab, copy your region name (not region ID) from the Regions table to use for your ASTRA_DB_REGION environment variable. For example, us-central1.

    Astra Regions 
    export ASTRA_DB_REGION=<region ID>

  3. Click Generate Token. Copy the token for your ASTRA_DB_APPLICATION_TOKEN environment variable.

    Generate Token 
    export ASTRA_DB_APPLICATION_TOKEN=<application token>

  4. Set the ASTRA_DB_NAMESPACE to default_keyspace.

    export ASTRA_DB_NAMESPACE=default_keyspace

  5. In the OpenAI Platform, get your API key to use for your OPENAI_API_KEY environment variable.

    export OPENAI_API_KEY=<OpenAI API key>

    You should now have the following environment variables set for your database:

export ASTRA_DB_ID=<database ID>
export ASTRA_DB_REGION=<database region>
export ASTRA_DB_APPLICATION_TOKEN=<application token>
export ASTRA_DB_NAMESPACE=default_keyspace
export OPENAI_API_KEY=<OpenAI API key>

Set up the code

  1. Clone the chatbot repository and switch to that directory.

    git clone https://github.com/datastax/astra-db-chatbot-starter.git \
  && cd astra-db-chatbot-starter

  2. Verify that pip is version 23.0 or higher.

    pip --version

    Upgrade pip if needed.

    python -m pip install --upgrade pip

  3. Install Python dependencies and load the data you will use in this tutorial.

    pip3 install -r requirements.txt
    python3 populate_db/create_collection.py

  4. Run the load_data.py script to load the scraped website data that lives in scraped_results.json.

    python3 populate_db/load_data.py

    OpenAI has rate limits, which could affect how effectively you can complete this tutorial. For more, see OpenAI’s rate limits.

    scraped_results.json has the following structure:

    scraped_results.json
    {
    [
        {
            "url": url_of_webpage_scraped_1,
            "title": title_of_webpage_scraped_1,
            "content": content_of_webpage_scraped_1
        },
        ...
    ]
}

    For more about the embeddings, see About the tutorial embeddings.

Send prompts to OpenAI with LangChain and FastAPI

  1. Install the web server for the chat API.

    pip install uvicorn

  2. Start the web server.

    uvicorn api.index:app --reload

    The API is split between a few files:

    • index.py contains the actual FastAPI code. It defines an API with a single route: /api/chat. Upon receipt of a POST request, the function calls the send_to_openai function from chatbot_utils.py.

    • chatbot_utils.py loads authentication credentials from local_creds and connects to your database.

    OpenAI Diagram

  3. Using your application token, connect to the database by running chatbot_utils.py.

    python3 api/chatbot_utils.py

    If the request works correctly, no response is returned.

    chatbot_utils.py sends the prompt to OpenAI. The definitions leave space that is later filled in with the user’s question and the relevant documents.

    #prompt that is sent to OpenAI using the response from the vector database and the users
original query
prompt_boilerplate = ("Answer the question posed in the user "
                      "query section using the provided context")
user_query_boilerplate = "USER QUERY: {userQuery}"
document_context_boilerplate = "CONTEXT: {documentContext}"
final_answer_boilerplate = "Final Answer: "

    These are the required functions:

    • get_similar_docs searches for the most similar documents in the database. This guarantees that you pull information relevant to the question from our database.

    • build_full_prompt extracts the actual text from the LangChain documents returned from get_similar_docs. It takes that text and the original user-provided question and combines them with the prompt template defined above to construct the prompt that is sent to the GPT model for completion.

    • send_to_openai sends a prompt to the GPT model using the .prediction method from LangChain’s OpenAI class. Whatever the model returns is the result of the function.

  4. Test your API by sending it a request.

    curl --request POST localhost:8000/api/chat \
  --header 'Content-Type: application/json' \
  --data '{"prompt": "Can encryptions keys be exchanged \
          for a particular table?" }'

    The endpoint returns the following response:

    "{\"text\": \"\\nNo, encryptions keys cannot be exchanged for a particular table. However, you can use the GRANT command to configure access to rows within the table. You can also create an entirely new keyspace with a different name if desired.\", \"url\": \"https://docs.datastax.com/en/dse68-security/docs/secFAQ.html\"}"

    This response is a string, so you may need to convert it to JSON.

Deploy your chatbot

To see the chatbot in action, you can deploy it to a serverless environment, such as Vercel.

  1. In the Vercel Dashboard, search for and import the third-party Git repo from https://github.com/datastax/astra-db-chatbot-starter.

  2. Select the Next.js Framework Preset option.

  3. Set the Environment Variables to match the ones you defined above.

  4. Click Deploy.

After a few minutes, you will see your deployed chatbot app.

Chatbot app

After you deploy in Vercel the first time, auto-deployment will trigger for each subsequent commit.

For more about using Vercel, see the Vercel documentation.

About the tutorial embeddings

We define our embedding model to match the one used in the vectorization step. We use LangChain’s OpenAI library to create a connection to their API for completions as well as embeddings.

The load_data.py script loads the scraped website data that lives in scraped_results.json with the following structure:

scraped_results.json
{
    [
        {
            "url": url_of_webpage_scraped_1,
            "title": title_of_webpage_scraped_1,
            "content": content_of_webpage_scraped_1
        },
        ...
    ]
}

If you’d like to learn more about how this data is scraped, you can find the scraper and instructions in the README.

This is a portion of the text scraped from the Datastax Premium Support FAQ page:

input_json = {
    "url": "https://www.datastax.com/services/support/premium-support/faq",
    "title": "FAQ | DataStax Premium Support",
    "content": (
        "What is DataStax Premium Support?\n"
        "Premium Support consists of three add-on offerings available to enhance your existing DataStax support experience\n"
        "There are three Premium offerings currently available:\n"
        "Premium Cloud Engineer\n"
        "Named Engineer\n"
        "Technical Account Manager\n"
        "Can I purchase a Premium Support subscription for any DataStax product? \n"
        "You can purchase Premium Support as an add-on to DataStax products that are generally available. Please contact PremiumSupport@datastax.com if you have any questions.\n"
        "What is included in each of the different DataStax technical support offerings?"
        # ...
    )
}

The text in input_json['content'] is split on the newline character. The lines that are questions are stored in question_lines.

all_lines = [
    0["What is DataStax Premium Support?"]
    1["Premium Support consists of three add-on offerings available to enhance your existing DataStax support experience"]
    2["There are three Premium offerings currently available:"]
    3["Premium Cloud Engineer"]
    4["Named Engineer"]
    5["Technical Account Manager"]
    6["Can I purchase a Premium Support subscription for any DataStax product?"]
    7["You can purchase Premium Support as an add-on to DataStax products that are generally available. Please contact PremiumSupport@datastax.com if you have any questions."]
    8["What is included in each of the different DataStax technical support offerings?"]
    # ...
]

question_lines = [0, 6, 8] # ...

There are questions on lines 0, 6, 8 …​ This means that the answers will be on lines 1 to 5, 7 …​

Now the answer lines are joined into a list and the output data structure is defined:

q_and_a_output_data = {
    "url": "https://www.datastax.com/services/support/premium-support/faq"
    "title": "FAQ | DataStax Premium Support"
    "questions": [
        ["What is DataStax Premium Support?"],
        ["Can I purchase a Premium Support subscription for any DataStax product?"],
        # ...
    ]
    "answers": [
        [(
            "Premium Support consists of three add-on offerings available to enhance your existing DataStax support experience\n"
            "There are three Premium offerings currently available:\n"
            "Premium Cloud Engineer\n"
            "Named Engineer\n"
            "Named Engineer\n"
            "Technical Account Manager"
        )],
        [(
            "You can purchase Premium Support as an add-on to DataStax products that are generally available. "
            "Please contact PremiumSupport@datastax.com if you have any questions."
        )]
        # ...
    ]
}

Once the text is split into answers and questions, each question and answer pair is iterated. The concatenated text of each question and answer is embedded with the OpenAI text-embedding-ada-002 model.

The first iteration through the question and answer data gives the following result:

question_id = 1
document_id = "https://www.datastax.com/services/support/premium-support/faq"
question    = "What is DataStax Premium Support?"
answer      = (
    "Premium Support consists of three add-on offerings available to enhance your existing DataStax support experience\n"
    "There are three Premium offerings currently available:\n"
    "Premium Cloud Engineer\n"
    "Named Engineer\n"
    "Technical Account Manager"
)
# (The embedding is a 1 x 1536 vector that we've abbreviated)
embedding   = [-0.0077333027, -0.02274981, -0.036987208, ..., 0.052548025]

At each step, the data is pushed to the database:

to_insert = {
    "insertOne": {
        "document": {
            "document_id": document_id,
            "question_id": question_id,
            "answer":answer,
            "question":question,
            "$vector":embedding
        }
    }
}

response = requests.request(
    "POST",
    request_url,
    headers=request_headers,
    data=json.dumps(to_insert)
)

The collection is now populated with documents containing the question and answer data and the vector embeddings.

Conclusion

Congratulations! In this tutorial, you completed the following:

  • Created a vector database.

  • Added data scraped from our FAQ pages.

  • Used the scraped data to add context to prompts to the OpenAI API.

  • Held a chat session with an LLM.

You can now customize the code to scrape your own sites, change the prompts, and change the frontend to improve your users' experience.

See also

If you want to get hands-on with a vector database as quickly as possible, begin with our Quickstart.

You can also learn more about Beginner’s guide to vector databases and Guide to embeddings.

Support

Was This Helpful?

Need More Help?

groupAsk the Astra community supportGet in touch with support

Give Feedback

How can we improve the documentation?