The Code, Brain of AIs: A Revolution in Their Functioning

Code, the Pillar of AI Agents' Reasoning

A recent study conducted by researchers at the University of Illinois Urbana-Champaign, in collaboration with Meta and Stanford, sheds light on a fundamental shift in how artificial intelligence (AI) agents operate. According to this research, computer code is no longer just a byproduct of AIs; it has become the very foundation of their ability to reason, act, and interact.

This change is largely attributed to an innovative software layer called the "harness." This system provides AI agents with tools and isolated environments, transforming them from stateless models into dynamic systems capable of planning, executing, and testing in a continuous cycle.

The Crucial Role of the "Harness"

Commercial systems such as Claude Code and Codex from OpenAI are already leveraging this approach. However, the authors of the study caution against over-reliance on these systems. Current software testing methods are often incomplete, which can obscure potential risks. Therefore, more transparent and rigorous evaluation mechanisms are needed to ensure the reliability of these systems.

Why Code is Essential for AI Agents

The authors of the study highlight several reasons why code has become an essential component of AI agents' behavior:

• Executability: Code allows the transformation of model outputs into verifiable operations.
• Traceability: Intermediate calculations can be tracked through structured traces, facilitating data reading and storage by the system.
• Persistence: Running code records task progress, enabling agents to resume where they left off.

The article divides long-term agent systems into three main components: the intrinsic capabilities of the model (such as reasoning and planning), the infrastructure provided by the system, and the code generated by the agent itself, ranging from test scripts to reusable skills.

Three Levels of Code Organization

The first level of code organization connects the model to its environment. Methods like Program-of-Thoughts or Chain of Code externalize calculations to executable programs, rather than simply describing them in natural language. Other systems, such as Code as Policies, directly convert natural language instructions into control code for robots.

The reliability of these systems relies on clearly regulated state transitions within a controlled loop around the model.

Maintaining the Reliability of AI Agents

The second level concerns the reliability of agents across multiple stages. This includes planning, memory, tool usage, and a recurring cycle of planning, execution, and verification. This cycle replaces ad-hoc troubleshooting with systematic checks.

The third level focuses on collaboration among multiple agents. Collections of code, tests, and execution logs become a shared workspace where specialized roles, such as managers, planners, coders, reviewers, and testers, divide the work.

Concrete Industry Examples

The authors cite examples of commercial products that follow this model. Claude Code from Anthropic integrates the local terminal, development environment, and browser into a single workflow, where the agent edits files, executes commands, and must adhere to permission rules. The coding agents of Codex from OpenAI and GitHub Copilot shift similar workflows to managed cloud environments, consolidating changes through traceable pull request outputs.

The importance of this layer was unexpectedly highlighted when Anthropic accidentally leaked about 500,000 lines of source code from Claude Code. Among these lines was a "dream" function for task consolidation and other tricks to guide models as coding agents.

The Future of the "Harness" in AI Systems

Other AI labs are beginning to recognize the importance of the "harness." Deepseek plans to compete with Claude Code and Codex with its own product, Deepseek Code, and is building a dedicated "Harness" team in Beijing to manage everything beyond the model, from tool usage to planning and storage.

Optimization and Challenges of the "Harness"

Several research systems are focused on optimizing the harness itself. AutoHarness automatically generates code that filters unauthorized actions, while Meta-Harness systematically explores better harness variants using previous versions, their evaluations, and execution logs as a research space.

However, the authors highlight several open issues that hinder the field: more meaningful evaluations beyond raw success rates, verifying the substance of results when testing alone is insufficient, autonomous improvement of the harness without regressions, shared state among multiple agents, human oversight, and extension to environments with image or sensor data like GUI agents and robots.

They particularly emphasize the need to revisit current testing criteria. Tests can be incomplete, and test programs for graphical interfaces may lack poor intermediate steps. Simulators mask physical risks. A harness could engender false confidence precisely because it provides visible feedback, and the green checkmark does not mean the code is safe. The authors suggest that every accepted action should be accompanied by documentation specifying which tests were actually performed, which areas remained untested, and what risks persist.

The reliability of autonomous coding agents does not stem from better repair prompts, but from closely regulated state transitions within a controlled loop around the model, the researchers argue.