Claude Code and Codex: AI Loops and Firefox Bugs

Understanding AI Agent Loops: Concepts and Applications in Claude Code and Codex

In a practical guide, Claire introduces us to the world of AI agent loops, explaining concepts such as heartbeats, crons, hooks, and goal-based loops. She demonstrates how these elements can be integrated into Claude Code and Codex to automate various tasks, such as daily PR reviews or weekly creation of sub-agents for specific skills. This tutorial aims to demystify loop engineering, often perceived as complex, by making it accessible to novices.

The Fundamentals of Loops

A loop, in this context, is essentially a prompt that triggers automatically. Although the term may sound intimidating, it is merely a form of basic automation, popularized by the media cycle. Heartbeats, crons, and webhooks are concepts that have existed for a long time, but their application to AI agents is relatively new. This allows for a shift from batch processing to continuous interaction with AI.

The Power of Goal Loops

Goal loops represent the most powerful form of these automations, although they are often misunderstood. They define a specific outcome and continue executing an agent until that outcome is achieved or the agent encounters an obstacle. Unlike timer-based loops, they stop only when the work is completed. To prevent the agent from looping indefinitely and consuming tokens, it is advisable to let Codex draft its own objectives, relying on OpenAI's goal-writing guide.

Comparison with Employee Onboarding

Loops can be compared to onboarding a new employee. It involves clearly defining the work: what needs to be checked, how often, the expected outcome, and contacts in case of issues. For example, a loop prompt could be: "Every Friday at 10 AM, review all merged PRs and identify missing skills from our agents."

Generation of Sub-Agents

One of the most powerful aspects of loops is their ability to generate sub-agents. In Claude Code, the PR review loop not only checks the status of PRs; it creates dedicated sub-agents to monitor each PR until all checks are positive. Similarly, in Codex, a skills loop identifies gaps and immediately generates sub-agents to validate each new skill.

Cost and Efficiency of Loops

Loops can become costly if not carefully designed. Vague success criteria or a low validation threshold can lead to the agent running continuously without significant progress. It is crucial to monitor both the cost and the quality of the output from the outset.

Using Simple Loops

A morning briefing in Claude Cowork is an example of a simple and effective loop. A scheduled task that triggers every morning, checks the calendar and emails, and sends a summary to Slack constitutes a functional loop without requiring code. From there, it is easy to move on to more complex tasks like PR reviews or skills identification in Claude Code or Codex.

Strategic Automation

The strategy is to create loops capable of generating their own sub-agents. In the Codex demonstration, the weekly automation generated two named sub-agents, each executing its own goal loops to validate skills in real-time. The potential of loop-based automation depends on how clearly the work is defined, rather than the complexity of the engineering.

Claude Mythos and the Revolution in Bug Detection in Firefox

Brian Grinstead, an engineer at Mozilla, shares how his team used AI agents to fix 423 security bugs in Firefox in one month. The success lies not only in a better model but in the customized harness that accompanies it: assessing files, executing goal loops, verifying bugs with sub-agents, all while maintaining human intervention in the review process.

Harness and Efficiency

The spike in security bug fixes in Firefox is attributed not only to the Mythos model but especially to the harness. The latter provides AI agents with the necessary tools to identify, verify, and fix bugs. Brian explains that it is a simple yet effective wrapper that gives access to the right tools for the job.

Persistence of Agents

AI agents possess a persistence that humans cannot match. They can try 14, 15, or even 20 different approaches to trigger a bug without tiring. Brian discovered bugs that required 14 attempts before being resolved. Unlike humans, agents do not experience a decline in cognitive energy over time.

Verification and Reduction of False Positives

The verification loop is essential for eliminating false positives. Firefox employs a two-step process: the agent must first trigger a crash in their fuzzing build, then a verifying sub-agent ensures that the bug report is relevant and not just related to test configurations. Thus, when the bug reaches human engineers, it is nearly free of false positives.

Limitations of Agents

While agents are effective at fixing specific bugs, they may lack a broader perspective. Often, when an agent fixes a bug, it merely patches the vulnerable location. Human engineers must then review the fix and ensure that other similar areas in the code are also checked.

Task Prioritization

With millions of lines of code, prioritization is crucial. Firefox has developed a simple LLM judge that evaluates each file based on two criteria: the likelihood of a memory security issue and the ease of access from a web page. Brian emphasizes that this system is simple and reproducible by anyone.

Building a Harness

Building a harness can be accomplished in an afternoon using provider SDKs. Firefox started with the Claude agent SDK, which is a wrapper around the Claude Code CLI, streaming JSON and providing programmatic hooks. Brian advises using the harnesses provided by the vendor, as models are often post-trained to work best with their own infrastructure.

Diversity of Models and Approaches

For security work, it is recommended to run multiple models and harnesses. Attackers use various techniques to find bugs, so defenders must scan with multiple approaches. Different models and harnesses focus on different strengths and identify various vulnerabilities.

Beyond Security

This approach is not limited to security: performance, technical debt, and user experience are also viable targets. The same pattern applies: evaluate and prioritize areas of your codebase, give the agent a constrained objective with verification criteria, and integrate the results into your existing pipeline. Brian mentions that they are actively working on performance optimization using the same harness structure.