Google equips its AI agents with persistent memory to advance

A Google Research executive associates AI agents with a knowledge wiki that enriches with each execution. Notable increases have been observed across several benchmarks, especially in math and spreadsheets, and even more so with large models. Skills are reversible in case of underperformance and sometimes transferable between models, with varying success depending on the tasks.

Announced Gains, Highly Variable by Task

Researchers indicate that WikiSkill consistently outperforms other skill evolution approaches and the baseline without skills. On average, Gemini-3.5-Flash improves from 49.5% to 68.1%, while Qwen-3.6-27B progresses from 39.4% to 63.3%. On individual benchmarks, Gemini-3.5-Flash rises from 33.0% to 72.6% on LiveMath and from 50.5% to 76.6% on SpreadSheet. The most significant increases concern mathematical problems and spreadsheet manipulation, while tasks with long document contexts, such as OfficeQA, show only minimal progress. The gap from the baseline increases with model size.

Effect of Model Size and Transferable Skills to Verify

Larger models tend to derive greater benefits from evolved skills. Smaller models incorporating WikiSkill can achieve the same results as larger models lacking this framework. Skills acquired by one model are frequently transferred to another, sometimes exceeding those that the recipient model had developed on its own; this transfer capability is not systematic and should be verified individually. Conversely, compact models like Qwen-3.5-4B struggle to reliably apply multi-step search strategies on long contexts and revert to their default behavior. Evaluations focused on Qwen (4B, 9B, 27B), Gemma-4-31B, and Gemini-3.5-Flash.

A Documented Execution Cycle, Skill Proposal, and Filtering

The process begins with an inference agent executing tasks based on its current skills and producing complete traces. A Wiki Maintainer inspects these traces, isolates failure patterns and effective strategies, and then records them in a wiki. A Skill Proposer then leverages this updated wiki and execution data to suggest targeted skill adjustments. Each proposal undergoes a filtering mechanism that tests it on a separate validation set to confirm its benefit. If the skill fails, it is downgraded, but the wiki content remains unchanged; these inconclusive trials are also logged to guide future iterations. The evaluation covered five benchmarks, including mathematical reasoning, web search, spreadsheet usage, document question-answering, and interactive tasks.

The Architecture Distinguishes Experience, Knowledge, and Action on Three Levels

The architecture separates experience, knowledge, and action across three levels. The Raw Layer retains immutable execution logs ranging from tool calls to results. Above it, the Wiki Layer condenses this data into structured elements, and according to researchers, its content never resets and continues to grow. Finally, the Skill Layer groups procedural instructions that the agent implements during its tasks; these skills can be revoked if an update degrades performance.

No Continuous Learning, but Refining Instructions

WikiSkill associates AI agents with a persistent knowledge base and groups acquired knowledge into Agent Skills, reusable modules that guide behavior without altering learned parameters. The agent does not perform continuous learning; it produces and reuses higher-quality instructions with each execution. This approach is deemed effective, although it may be less elegant and more prone to errors than true learning. This bias is inspired by a reflection from Andrej Karpathy around a "LLM Wiki" aimed at transforming experience into cumulative and persistent knowledge.