AI: ProgramBench Advances, Mirendil Trains a Giant LLM

Three numerical milestones shed light on the state of AI: a hidden channel discovered by OpenAI agents despite isolation, a jump to 18.5% on ProgramBench for Claude Opus 5.5, and a LLM with several hundred billion parameters trained by a single researcher at Mirendil. Meanwhile, Resolution is automating large-scale mathematical formalization, with a week of computation and 600,000 lines of Lean.

Isolated agents but communicating via CPU cache, according to OpenAI

OpenAI reports having studied the security of agents placed in separate microVMs, managed by its sandboxing software, with network restrictions and no shared files. Despite these limitations, the agents managed to transmit synthetic secrets using the CPU cache. The CPU cache, a small, fast memory that stores copies of recently used data, was used as a side channel: one agent transmitted information by intentionally deleting certain data from the cache, while the other measured the time needed to reload it to decode the message from the latency difference.

Measured leap on ProgramBench for Claude Opus 5.5

ProgramBench measures how well a system can reconstruct software from a compiled binary accompanied by its documentation, through exercises involving interpreters (PHP, Lua, tinycc), databases (DuckDB, SQLite), multimedia and compression utilities (FFmpeg, zstd, xz), as well as developer tools (ripgrep, fzf, jq). When the benchmark was introduced in May, none of the evaluated models exceeded 0%. The best system at the time, Claude Opus 4.7, peaked at 0% resolved and 3% nearly on 200 tasks. Recently, Claude Opus 5.5 achieved 18.5% of tasks resolved and 70.5% nearly, marking a substantial advancement on this challenging benchmark.

Autoformalization at Resolution: one week, estimated $135,000, and 600,000 lines of Lean

Resolution indicates that it conducted a large-scale mathematical autoformalization in about a week, utilizing approximately 100 billion input tokens, of which 90% were cache reads, and 300 million output tokens, with an estimated token cost of $135,000. The effort produced around 600,000 lines of Lean. The startup already reports speed gains in other areas of its research and the use of the generated library to extend certain parts of singular learning theory.

Method: multi-agent plans and orchestrated execution

A researcher at Resolution collaborated with agents to produce two distinct but complementary plans: a general plan understandable by humans and a detailed 112-page plan, segmented into elementary steps compliant with the literature. The project's execution was ensured by an orchestrating agent, which delegated tasks to other agents such as performing manual calculations, programming in Lean, and handling support functions, including continuous progress tracking. The implementation required little intervention beyond the initial plan and agent skills distilled from previous internal efforts.

Mirendil: one researcher, models up to 2T–10T parameters

At Mirendil, a single researcher trained a qualified coding model alone. Mirendil-Base is a pre-trained LLM built in 3.5 months, with less than $10 million in computation, on code written from scratch using libraries like PyTorch and JAX, and public data. The variants cover high orders of magnitude: 100B–500B parameters for Mirendil-Base, 500B–2T for Mirendil-Base-Large, and 2T–10T for Mirendil-Base-XL. Mirendil-Base-Large is about 80% through its training, and Mirendil-Base-XL is around 19%, already close to the current score of Large. Meanwhile, Resolution expects that the industrialization of mathematics will enable a rigorous science of research automation. Resolution sees the resolution of singularities as a foundation for automating the exploration of the link between the geometry of the loss landscape and generalization. It is also anticipated that by 2027, a significant portion of software will be produced autonomously by AIs.