Outlines Revolutionizes Structured Generation with LLMs

A New Era for Language Models with Outlines

Large Language Models, or LLM, are often called upon to produce structured outputs, such as JSON objects. However, achieving perfect structure typically requires careful phrasing of queries and a bit of luck. This challenge persisted until the arrival of an innovative open-source library: Outlines.

Outlines was designed to address the typical issues faced by LLMs when generating structured outputs, particularly hallucinations. It introduces a level of deterministic certainty into the generation process, ensuring more reliable results.

This article explores the capabilities of Outlines through practical examples in Python, demonstrating how it can transform interactions with LLMs.

Sentiment Analysis: Accurate Classification

Before diving into the first use case, it is essential to understand how Outlines ensures the accuracy of structured outputs from models. During inference, it masks "syntactically illegal" tokens during generation, rather than attempting to correct poorly formulated text once generated. This prevents the model from violating the rules of the desired output format.

Let's take the example of a pipeline for analyzing customer support tickets, where the goal is to select an option from a limited, approved list. This is a classification problem, and Outlines' generate.choice() function forces the model to choose from predefined literals.

To get started, you need to install Outlines with transformers to access pre-trained LLMs:

pip install outlines[transformers]

The following code uses outlines.from_transformers() to load a pre-trained model, aided by the automatic classes from Hugging Face. The model and its tokenizer are wrapped in an Outlines object, which guides the model on the expected outcomes. During inference, the user query is accompanied by a Literal object defining the output constraints:

from transformers import AutoTokenizer, AutoModelForCausalLM
from typing import Literal

# 1. Load the backend using Transformer-based models
model_name = "microsoft/Phi-3-mini-4k-instruct"

# We use outlines to load the model with its from_transformers() function
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(model_name),
    AutoTokenizer.from_pretrained(model_name)
)

# 2. Call the model directly, passing our approved strings as type constraints
sentiment = model(
    "Classify the sentiment of this customer review: 'I've been waiting two weeks for my delivery and it's still missing.'",
    Literal["Positive", "Negative", "Neutral"]
)
print(sentiment)

Although the literal is part of Python's built-in typing module, Outlines takes control of the model, enforcing standard Python types and building a finite state machine in the background that limits the output to only the provided options.

Generating JSON Objects: Guaranteed Structure

In this example, a Pydantic object is first defined to specify the structure of a JSON object describing a fictional character with a name, description, and age. The already wrapped Outlines model is used to ensure that the generated output adheres to this structure:

from pydantic import BaseModel

# 1. Define a Pydantic model for the desired JSON structure
class Character(BaseModel):
    description: str

# 2. Use the model wrapped by outlines to generate a JSON output conforming to the Pydantic model
json_output = model(
    "Generate a JSON object describing a fictional character named 'Anya'.",
    max_new_tokens=200
)
print(json_output)

{
    "name": "Anya",
    "description": "Anya is a young, adventurous woman with a passion for exploring new places and meeting new people. She has long, curly hair and bright green eyes that sparkle with curiosity. Anya is always eager to learn and loves to share her knowledge with others. She is kind-hearted and always willing to lend a helping hand to those in need. Anya's favorite hobbies include hiking, reading, and playing the guitar. She is a free spirit who values freedom and independence above all else.",
    "age": 25
}

Generating JSON for REST APIs: Increased Precision

The third example, also focused on JSON, takes place in a different context. Imagine creating an API that requires a well-defined JSON payload to update a database. Standard LLMs may produce unwanted characters, such as commas, that could cause a JSON parser to fail.

With Outlines, the JSON payload schema is defined using a custom class object based on Pydantic:

from pydantic import BaseModel
from typing import Literal

class ServerHealth(BaseModel):
    service_name: str
    uptime_seconds: int
    status: Literal["OK", "DEGRADED", "DOWN"]

# 1. Outlines should produce a raw string guaranteed to be valid JSON
raw_json_string = model(
    "Report the current status of the main Auth database.",
    max_new_tokens=50
)
print(type(raw_json_string))  # This will simply display:

# 2. Formatted print
parsed_json = json.loads(raw_json_string)
print(json.dumps(parsed_json, indent=2))

{
    "service_name": "auth_db_status",
    "uptime_seconds": 1623456789
}

Conclusion: A Leap Forward for Structured Outputs

LLMs, designed to mimic human conversations, can often break syntax or produce hallucinations to seem more natural. Obtaining reliable and structured outputs, such as clean JSON objects, can be a challenge. Outlines, as an open-source library, introduces deterministic certainty into the output generation process of LLMs, thereby enhancing the reliability and accuracy of structured outputs. This article illustrated three simple yet effective use cases for beginners with this promising tool.

Iván Palomares Carrascosa is an expert in AI, machine learning, deep learning, and LLMs. He trains and guides others in applying AI in real-world contexts.