LLM 0.32a0: Revolutionizing Python Model Management

LLM 0.32a0: A Major Overhaul

The alpha version 0.32a0 of the LLM Python library represents a significant transformation in how we interact with language models. This update is a major redesign of the LLM Python library and the CLI tool for accessing LLMs, while remaining compatible with previous versions. Since April 2023, LLM has evolved to adapt to the rapid advancements in the field of language models, necessitating a redesign to better handle various types of inputs and outputs.

History and Feature Evolution

Initially, LLM operated on a simple model based on prompts and text responses. You would send a text to the model and receive a text response, as illustrated by the following example:

model = llm.get_model("[gpt](/glossaire/gpt)-5.5")
response = model.prompt("Capital of France?")
print(response.text())

However, this approach became insufficient as models evolved to support image, audio, and video inputs, as well as structured JSON schemas. LLM had to adapt to integrate these new capabilities. LLM provides an abstraction over thousands of different models through its plugin system. This original abstraction, which consisted of a text input returning a text output, was no longer capable of representing everything I needed.

New Interaction Mechanisms

Version 0.32a0 introduces two major changes. First, inputs can be represented as a sequence of messages, a method inspired by the bidirectional conversational interface of ChatGPT. This allows interactions to be structured as continuous dialogues, facilitating the emulation of chat completion APIs.

For example, a conversation could start with:

user: Capital of France?

And continue with:

user: Capital of France?
assistant: Paris

The JSON APIs of major providers follow this model, as shown in the example with OpenAI's chat completion API:

curl https://api.openai.com/v1/chat/completions \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
"model": "gpt-5.5",
"content": "Capital of France?",
"role": "assistant",
"content": "Paris",
"content": "Germany?"

Before version 0.32, LLM modeled this as conversations but did not allow for resuming an already started discussion. Now, thanks to the use of functions like llm.user() and llm.assistant(), it is possible to structure complex conversations from the outset. The llm.user() and llm.assistant() functions are new construction functions designed to be used in the messages=[] array. The previous prompt= option still works, but LLM updates it to a single-element messages array in the background. You can also now respond to a response, as an alternative to building a conversation.

The CLI tool of LLM circumvented this with a custom mechanism to persist and deploy conversations using SQLite, but this never became a stable part of the LLM API — and there are many places where you might want to use the Python library without committing to using SQLite as a storage layer.

Streaming Results

The other major novelty of this version is the streaming of results. Previously, LLM supported streaming in a basic way, but the new version allows for processing mixed data streams, including text, tool calls, and potentially images or audio snippets. This is crucial for models capable of returning varied content.

The new alpha LLM models streaming as a flow of typed message parts. Here’s an example of the new streaming interface:

model = llm.get_model("gpt-5.5")
prompt = "invent 3 cool dogs, first talk about your motivations"
def describe_dog(name: str, bio: str) -> str:
    """Record the name and biography of a hypothetical dog."""
    return f"{name}: {bio}"
def sync_example():
    response = model.prompt(
        tools=[describe_dog],
        for event in response.stream_events():
            if event.type == "text":
                print(event.chunk, end="", flush=True)
            elif event.type == "tool_call_name":
                print(f"\nTool call: {event.chunk}(", end="", flush=True)
            elif event.type == "tool_call_args":
                print(event.chunk, end="", flush=True)

Serialization and Deserialization Mechanism

Version 0.32a0 also introduces a new serialization and deserialization mechanism for responses, offering users the ability to create custom alternatives for conversation persistence. Here’s how it works:

serializable = response.to_dict()
# serializable is a JSON-style dictionary
# store it wherever you want, then deploy it:
response = Response.from_dict(serializable)

The returned dictionary is actually a TypedDict defined in the new module llm/serialization.py.

Future Perspectives

I am releasing this as an alpha to be able to update various plugins and test the new design in real-world environments for a few days. Although this version is still in alpha, it foreshadows a stable version that should retain these advancements unless testing reveals design flaws. There remains an important task: redesigning the SQLite logging system to better capture the finer details returned by this new abstraction. Ideally, I would like to model this as a graph, to better support situations like an OpenAI-style chat completion API where the same conversations are constantly extended and then repeated with each prompt. I want to be able to store this without duplicating it in the database. This update represents a significant step towards a more flexible and powerful use of language models, meeting the growing needs of developers and advanced users.