Build a recommendation system with vector search, LangChain, and OpenAI

query_builder 25 min

A vector search runs to display similar products to the user. In this tutorial, you will build a simple product recommendation system. As products are added or updated, the embeddings in the database are automatically updated. Using a large language model (LLM) and vector search, you do not have to manually categorize the products. The embeddings determine similar products, reducing the effort to manage the recommendations.

Prerequisites

For this tutorial, you need the following:

An active Astra account
An OpenAI account with a positive credit balance
A Vercel account
Python 3.9 or later
Node.js 18 or later
git

You should also be proficient in the following tasks:

Interacting with databases.
Running a basic Python script.
Entering parameters in a user interface for Vercel.

Clone the Git repository

Clone the chatbot Git repository and switch to that directory.

git clone https://github.com/datastax/astra-db-recommendations-starter/
cd astra-db-recommendations-starter

Create your Serverless (Vector) database

This tutorial uses a Serverless (Vector) database to store the WikiChat app data and embeddings. If you’re new to vector databases, see What are vector databases?.

In the Astra Portal navigation menu, click Databases, and then click Create Database.
Select the Serverless (Vector) deployment type.
Enter a meaningful, human-readable Database name.

After you create a database, you can’t change its name.

Database names are permanent. They must start and end with a letter or number, and they can contain no more than 50 characters, including letters, numbers, and the special characters & + - _ ( ) < > . , @.
Select a Provider and Region to host your database.

On the Free plan, you can access a limited set of supported regions. To access Locked regions, you must upgrade your subscription plan.

To minimize latency in production databases, select a region that is close to your application’s users.
Click Create Database.

New databases start in Pending status, and then move to Initializing. Your database is ready to use when it reaches Active status.

Set your environment variables

Export the following environment variables. Keyspace is optional.

export ASTRA_DB_API_ENDPOINT=API_ENDPOINT
export ASTRA_DB_APPLICATION_TOKEN=TOKEN
export ASTRA_DB_KEYSPACE=default_keyspace # A keyspace that exists in your database
export OPENAI_API_KEY=API_KEY
export VECTOR_DIMENSION=1536

Set up the code

Install project dependencies.

npm install
pip3 install -r requirements.txt

Load the data you will use in this tutorial.
```
python3 populate_db/load_data.py populate_db/product_data.csv
```
OpenAI has rate limits, which could affect how effectively you can complete this tutorial. For more, see OpenAI’s rate limits.

For more about the embeddings, see About the tutorial.

Deploy your assistant locally

You can deploy the recommendations assistant to a local environment.

Start the backend server (API), which you installed as part of the requirements.
```
uvicorn api.index:app --reload
```
Because the backend server continues to run in the terminal, open a second terminal.
In a new terminal, start the web server, which you installed as part of the requirements.
```
npm run dev
```
Open http://localhost:3000 to view the shop website with recommendations in your browser.

Deploy your assistant with Vercel

You can deploy the recommendations assistant to a serverless environment, such as Vercel.

In the Vercel Dashboard, search for and import the third-party Git repo from https://github.com/datastax/astra-db-recommendations-starter.
Select the Next.js Framework Preset option.
Set the Environment Variables to match the ones you defined above.
Click Deploy.

After a few minutes, you will see your deployed recommendations assistant.

After you deploy in Vercel the first time, auto-deployment will trigger for each subsequent commit.

For more about using Vercel, see the Vercel documentation.

About the tutorial

Embeddings

The data for this tutorial come from a marketing sample produced by Amazon. It contains 10,000 example Amazon products in csv format, which you can see in the /populate_db folder. This tutorial loads each of these products into our Serverless (Vector) database alongside a vector embedding. Each vector is generated from text data that’s extracted from each product.

The original csv file has 28 columns: uniq_id, product_name, brand_name, asin, category, upc_ean_code, list_price, selling_price, quantity, model_number, about_product, product_specification, technical_details, shipping_weight, product_dimensions, image, variants, sku, product_url, stock, product_details, dimensions, color, ingredients, direction_to_use, is_amazon_seller, size_quantity_variant, and product_description.

It is important to store all of this information in your database, but not all of it needs to be processed as text for generating recommendations. Our goal is to generate vectors that help identify related products by product name, description, and price. The image links and stock count are not as useful when finding similar products. First, transform the data by cutting it down to a few fields that are good comparisons:

  input_rows = load_csv_file(filepath)
  for chunk_start in range(0, len(input_rows), EMBEDDING_CHUNK_SIZE):
      chunk = input_rows[chunk_start : chunk_start + EMBEDDING_CHUNK_SIZE]
      to_embed_strings = [
          json.dumps(
              {
                  key.lower().replace(" ", "_"): row[key]
                  for key in (
                      "Product Name",
                      "Brand Name",
                      "Category",
                      "Selling Price",
                      "About Product",
                      "Product Url",
                  )
              }
          )
          for row in chunk
      ]
      ...

Product name, brand name, category, selling price, description, and URL are the most useful fields for identifying the product. Convert the resulting JSON object to a string. Embed that string using the OpenAI embedding model "text-embedding-ada-002". This results in a vector of length 1536.

The embedding vectors are computed in chunks to optimize the overhead of calling the embedding API.

Add the returned vector to the full product JSON object and load it into the database collection. The insertion is done through the Python Data API client, which takes care of working in chunks if necessary.

      ...
      embedding_vectors = embed_list(to_embed_strings)
      documents_to_insert = []
      for row, embedding_vector in zip(chunk, embedding_vectors):
          document_to_insert = {
              "$vector": embedding_vector,
              **{key.lower().replace(" ", "_"): value for key, value in row.items()},
          }
          documents_to_insert.append(document_to_insert)
      insertion_result = collection.insert_many(documents_to_insert)

You now have your vector product database, which you can use for product search and recommendations.

Get embeddings for search and recommendations

The user performs a search with the search bar at the top of the user interface. An embedding is generated from the search query, and then used for a vector search on your product database.

The code removes the vector field from the search results, because it’s not needed to populate the catalog. The front-end displays the remaining fields.

Because you stored the vectors in the database, you don’t need to generate embeddings every time you get recommendations. Simply take the current product and use its ID to extract the vector from the database.

Backend API

The API is split between a few files:

index.py contains the actual FastAPI code. It defines an API with a few routes, which make use of functions for database retrieval, embedding computation and LLM usage.
recommender_utils.py manages the LLM-assisted recommendations and the search of similar product through embedding vectors.
query.py offers the primitive functions for product retrieval from the collection. It uses local_creds.py for the database authentication credentials and parameters.

Prompting to OpenAI with LangChain and FastAPI

recommender_utils.py sends the prompt to an LLM. The prompt template takes a search and relevant recommendations as parameters.

# prompt that is sent to OpenAPI using the response from the vector database
prompt_boilerplate = (
    "Of the following products, all preceded with PRODUCT NUMBER, select the "
    + str(count)
    + " products most recommended to shoppers who bought the product preceded "
    + "by ORIGINAL PRODUCT below. Return the product_id corresponding to "
    + "those products."
)
original_product_section = "ORIGINAL PRODUCT: " + json.dumps(
    strip_blank_fields(strip_for_query(get_product(product_id)))
)
comparable_products_section = "\n".join(string_product_list)
final_answer_boilerplate = "Final Answer: "
nl = "\n"
return (
    prompt_boilerplate
    + nl
    + original_product_section
    + nl
    + comparable_products_section
    + nl
    + final_answer_boilerplate,
    long_product_list,
)

These are the functions it uses:

strip_blank_fields removes empty-string fields from documents, which are uninteresting for the later LLM-assisted evaluation.
strip_for_query selects a subset of document fields, those known to be relevant for the LLM-assisted evaluation.

The response from the LLM includes a selection of the input products that were chosen to maximize the relevance to a given "original" product. In other words, the last step in refining the result set obtained by vector similarity search is made by an LLM.

Next steps

In this tutorial, you generated embeddings for Amazon products, stored those embeddings in a Serverless (Vector) database, and used prompt engineering to retrieve and display similar products based on a user’s search query. You also created an application to interact with this recommendation system.

Next, try customizing the code to add your own products, use different prompts, and redesign the frontend.

You can also try building apps with the Astra DB Data API clients. To get started with the clients, see the Astra DB Serverless quickstart for collections.