LLMs and Python: Optimizing Recommendations Without Excessive Costs

Large Language Models

Large Language Models, or LLMs, represent the current pinnacle of artificial intelligence capabilities. These models are trained on a vast dataset that encompasses all available knowledge in the world. However, their size and complexity mean that they are not often hosted internally by companies. Instead, they are typically accessed via APIs, which involves token consumption for each request, leading to significant costs.

Let’s take a concrete example to illustrate this point: if you want a recommendation for an Italian restaurant that is both romantic, well-located, and affordable, you might query a model like ChatGPT. However, if this model had to evaluate every possible restaurant to check if it meets your criteria, it would consume an enormous amount of tokens. This process would not only be costly but also slow, to the point that you might already be in bed before receiving a response.

Yet, it is essential not to overlook the ability of LLMs to interpret and extract relevant information. The key lies in the strategic use of these models, avoiding the need to leverage their full power at every step of the process.

1. System Design

In the field of engineering, the concept of the Precision-Scale-Time triangle is well known. It describes the necessary trade-offs between these three elements:

• A system can be very precise and capable of processing large amounts of data, but this will come at the expense of speed.
• It can be both precise and fast, but it will not be able to adapt to very large datasets.
• Finally, it can be fast and scalable, but it will then lack precision.

To achieve results that are both precise and efficient, it is possible to combine these approaches. Thus, a fast and scalable system can first provide a list of potential candidates, which will then be refined by an LLM to obtain more precise recommendations.

The strategy is therefore to:

• Conduct a quick and simple search to identify the K nearest restaurants, based on simple rules that ensure high recall but low precision.
• Then use an LLM to refine this list and choose the best restaurant according to the user's criteria, leveraging artificial intelligence for high precision.

This method allows for the benefits of LLM intelligence without incurring high costs, applying it only to a limited selection of candidates.

2. Data Generation

To recommend restaurants, it is first necessary to have a relevant database. In a real system, this would be done via a database stored in an S3 location. For this article, we use a synthetic database, allowing us to reproduce the process at no cost.

The RestaurantDataGenerator class in datagenerator.py generates a table of 10,000 restaurants spread across eight U.S. cities. Each restaurant is characterized by:

• a randomly generated name
• a geographical location (latitude/longitude) sampled around the center of each city, within a radius of about 13 km
• a type of cuisine (Italian, Japanese, Mexican, Thai, French, etc.)
• a dietary profile (omnivore, vegetarian, vegan)
• an average rating given by customers
• a number of votes cast
• a price range (10, 100, 1000, representing an average ticket per person).

This generator is designed to be executed only once, and the data is saved in data/restaurants.csv.

2.3 Candidate Generation

The first step in the process is to create a list of candidates based on simple and low-cost rules. The user specifies their city, and the system retains only the geographically closest restaurants. The code filters the table by city, calculates the direct distance between the user and each restaurant, and identifies the N_DISTANCE_CANDIDATES (50 by default).

This approach prioritizes high recall with lower precision, allowing for a review of all 10,000 restaurants without API calls or token costs. Although this method is not sophisticated, it eliminates non-viable options for the user, thus constituting a crucial first step.

For example, if you are looking for "cheap vegan tacos with a lively atmosphere" in several cities, this step will allow you to quickly filter out restaurants that do not meet these basic criteria.

2.4 Candidate Selection

The second step, which is more intelligent and precise, relies on the LLM. It builds on the list of 50 restaurants obtained previously. The LLM never processes all 10,000 restaurants, but only this restricted selection.

The model is queried via an OpenAI client, with an API key read from OPENAI_API_KEY. The recommender, defined as RestaurantRecommender, operates by processing the query and the city through RestaurantRecommender.recommender(query, city).

A few key points:

• Precision increases significantly. The initial step provided 50 nearby restaurants without distinction. The next step reads the specific query (for example, "cheap vegan tacos with a lively atmosphere"), eliminates irrelevant options, and retains only the top 5 to 10 choices with an accurate fit_score.

• Structured output with Pydantic. The model responds in a structured format, ensuring that each response adheres to a predefined schema.

The output schema includes the id and name of the restaurant, a fit_score between 0 and 100, and a brief justification. For example, for our three cities, the model reorganized the 50 candidates based on the requested criteria, highlighting vegan and Mexican restaurants with high scores.

Conclusion

The two-step process has proven effective in responding to the request for "cheap vegan tacos with a lively atmosphere" in three cities:

• The initial step provided a list of candidates with high recall but low precision.
• The subsequent step, using the LLM, refined these candidates to provide precise recommendations.

The final choices include:

• New York: Golden Spoon (vegan, 4.9) and Maison Fork (Mexican, affordable) with scores of 90 and 85.
• Miami: Royal Tavern & Co. (vegan, Mexican, affordable) with a score of 85.
• Boston: Urban Spoon and Little House, two budget-friendly Mexican options.