LLM 0.1a0: A Revolutionary Plugin for the OpenAI Chat API

An Innovative Plugin for the OpenAI Chat API

The plugin llm-chat-completions-server 0.1a0 has been developed to provide an endpoint for the OpenAI Chat Completions API. This plugin is part of the new content-addressable logs introduced in LLM 0.32rc1, which aim to efficiently manage OpenAI Chat Completion-type requests. These requests allow each incoming message to extend the previous conversation, thereby ensuring continuity in exchanges.

A typical request might look like this:

curl http://localhost:8002/v1/chat/completions \
-H 'Content-Type: application/json' \
"model": "qwen3.5-4b",
{"role": "user", "content": "Capital of France?"},
{"role": "assistant", "content": "Paris."},
{"role": "user", "content": "Germany?"}

In this context, the state of the conversation is tracked by the client, which gradually extends each request. The new schema design in LLM allows for deduplication of these messages through the use of hashes for the different parts of the message.

Setup and Operation

To test this system, the plugin can be installed and launched using the following commands:

uv tool install llm --pre
llm install llm-chat-completions-server
llm chat-completions-server -p 9001

This starts a localhost server on port 9001, which exposes all available LLM models through all installed plugins. This server uses an endpoint compatible with ChatGPT Completions, thus facilitating the integration of LLM models into applications using the OpenAI API.

It is noteworthy that GPT-5.6 Sol was originally responsible for writing the complete code for this plugin, demonstrating a deep mastery of the structure of the OpenAI Chat Completions API.