DeepMind: Dream-RSI Tests Strategies in Review

Google and DeepMind present Dream-RSI, a method that replays search histories to evaluate strategies without restarting computations. Trials with Gemini 3.1 Pro and 3.7 Flash show gains in coding, optimization, and GPU tasks with fewer attempts. Overly precise instructions can restrict exploration. The code is published on GitHub.

Overly precise instructions degraded a GPU task

Researchers evaluated a variant that condensed search histories into instructions meant to guide the agent. In a GPU task, this version displayed lower performance than one without explicit instructions. According to the researchers, excessively precise indications may limit the search space and hinder the agent's exploration of a broader range of approaches. Under Dream-RSI, this type of instruction is associated with less extensive exploration in the context of open-ended search tasks. The learned strategy adjusted the search effort: it initially reduced the number of attempts when performance improved, then increased it when progress stalled, coinciding with new gains. Anthropic CEO Dario Amodei recently warned about the pace of AI research, citing advancements in self-improvement.

Measured gains in coding, optimization, and GPU

Dream-RSI was tested with Gemini 3.1 Pro and Gemini 3.7 Flash across eight tasks divided into three domains, against a baseline with the same initial conditions and a fixed strategy. For a task aimed at developing the fastest program for a statistical calculation used in finance and genomics, the generated program outperformed sklearn and glmnet across all six test datasets. With Gemini 3.1 Pro, the average execution time decreased from 3,587 to 2,931 milliseconds, while the number of attempts dropped from 550 to 317. Dream-RSI also surpassed SimpleTES, which required 51,200 executions, while Dream-RSI only needed 317. For mathematical optimization and GPU kernel design tasks, the results were similar or superior, significantly reducing computational costs: in two GPU tasks, the number of executions was reduced by up to 2.43 times at the same performance level, and in two others, performance reached up to 2.09 times that of the baseline at equal budget.

Replaying the search tree without recalling the model

The Dream-RSI method reuses data from a completed search to test other strategies in the already explored space. During exploration, the agent records its attempts and their results, allowing an alternative strategy to be executed against these stored outputs without new live execution. This replay enables verification of what would have happened if other branches had been prioritized or abandoned earlier, without inventing new solutions and staying within the already constructed tree. The retained results allow for the examination of thousands of alternative policies without recalling the model or the evaluator, while changing the way of searching without touching the underlying model. This positioning addresses the limitations of classical approaches: a fixed strategy replays its dead ends, while online adaptation requires numerous attempts and the repetition of costly executions to evaluate multiple alternatives. Researchers from Google and DeepMind thus propose a method that reuses past searches to test strategies without incurring additional computational costs.

Place in the ecosystem: AlphaEvolve, AutoTTS, WikiSkill, Hyperagents

In 2025, Google DeepMind unveiled AlphaEvolve, a system where Gemini Flash produces code suggestions, Gemini Pro evaluates them, and an evolutionary algorithm retains the most effective versions. Dream-RSI operates at a more general stage, refining the overall search strategy. AutoTTS employs a programming agent to explore algorithms in a simulated environment; these approaches have outperformed manually designed methods while consuming fewer resources. Google Research recently introduced WikiSkill, which records successes and failures in a wiki and converts them into reusable instructions for the agent, while the follow-up analysis of Dream-RSI links explicit instructions to more restricted exploration on open tasks. Meta pushes further with Hyperagents, which allow agents to modify the mechanism driving their self-improvement. Recursive self-improvement is garnering increasing interest, and these methods stand out by the level at which they intervene and how they leverage execution history.

What this means for self-improving agents

Self-improving agents are envisioned to discover new algorithms, solve mathematical problems, or produce faster code by iterating through the propose, evaluate, learn, and retry loop over thousands of attempts. However, this dynamic confronts vast search spaces and exploration trade-offs between promising leads, parallel trials, and abandonments. The behavior of these systems depends on their ability to avoid repeated dead ends and manage the evaluation cost of alternative strategies. Researchers have published the code and additional details on GitHub, paving the way for more systematic reuse and comparisons by the community.