Cutting-Edge AI: Global Standards and the Driving Role of the United States

A laboratory focused on AI has outlined a roadmap to regulate automated research and recursive continuous improvement. It suggests that the United States should federate an international network of security institutes, common measures, and incident reporting protocols, without turning these standards into licenses. The overarching goal remains a beneficial AGI under human control.

Identified Risks, Including an Incident at Hugging Face and the Fear of Losing Control

An incident revealed at Hugging Face is cited as a glimpse into the potential misalignments that could occur if alignment and safeguards remain insufficient, while clarifying that it did not directly result from recursive continuous improvement. They believe that comparable risks could worsen as systems gain autonomy without robust safeguards. The feared scenario hinges on a practical loss of human control over AI development, rendering them unable to supervise opaque research processes, leading to more dangerous and less aligned systems. Meanwhile, increasing automation could significantly accelerate the pace of advancements, including through recursive improvement loops even with human involvement.

An International Framework to Avoid Fragmentation, Disorderly Action, and Unequal Capabilities

International safety and security standards for advanced AI are deemed as crucial as alignment research for mastering the technological frontier. These standards would aim for common definitions of evidence and a shared basis for the rigor of technical safeguards, clarifying what constitutes a credible mitigation of catastrophic risk. They would address three challenges: the fragmentation of assessments and reporting obligations between countries, complicating comparison and cross-border action; the lack of coordinated collective action, with the potential for undesirable outcomes when each nation acts alone while recursive improvement may exceed our evaluation capabilities; and the uneven distribution of skills and cutting-edge activities, which exacerbates these two issues. These observations apply equally to both open and closed models.

Mechanism 1: Network of Security Institutes and Common Technical Base, Without Approval Regime

The first proposed pillar: articulate complementary national and international standards leveraging existing AI security institutes in Australia, Canada, Germany, France, Kenya, Japan, South Korea, Singapore, India, and the United Kingdom. The framework would involve the CAISI and national industry bodies, focusing on frontier models and developers measured by capability benchmarks, and on managing the benefits and risks associated with automated research and recursive improvement. The United States could capitalize on the International Network for Measurement, Evaluation, and Advanced AI Science established in 2024, which brings together public institutions around measurement and evaluation. The standards arising from this work would serve as a technical foundation for measuring capabilities, assessing risks, and evaluating the sufficiency of safeguards. They would not constitute licenses, mandatory pre-approvals, or approval procedures, with each government deciding on their incorporation into its law.

Mechanism 2: Common Measures, Human Oversight, and Incident Protocols

The second pillar: define shared measures and incident reporting protocols to support collective action in the face of increasing autonomy in research and recursive improvement. The proposals focus on evaluating relevant progress and the share of autonomous research within a company, with a recent report cited as an initial contribution, as well as defining human oversight and thresholds triggering an immediate review. They also include a taxonomy of severity, reporting thresholds, and follow-up and response practices, backed by a reporting framework on disalignment characterized as early contribution. Finally, secure channels between critical infrastructure operators and governments are deemed necessary to share vulnerabilities, threats, and best practices, with a US-China dialogue presented as desirable at a time deemed opportune for discussions.

Governance, Competition, and Cooperation with Standardization Bodies

Various governance instruments are mentioned, including the CAISI, state legislation, a federal framework, and the establishment of new partnerships between the public and private sectors. The work on standards aims to maintain a competitive environment by soliciting input from designers of both open and closed models, independent experts, and the academic world, while ensuring transparency and impartiality towards businesses, states, and business models, without hindering the entry of new players or developers of open-weight models. This approach takes inspiration from the aviation and financial stability sectors, which have shared technical standards and cooperation mechanisms without transferring power. It would involve organizations such as ISO, the Frontier Model Forum, the Agentic AI Foundation, the Open Secure AI Alliance, and the Appia Foundation to connect international standards with concrete evaluations.

Final Goals and Expected Role of the United States

The United States is called upon to federate global technical standards for advanced AI, within a framework articulated around the two axes mentioned above. Each laboratory pursuing advanced capabilities is urged to take responsibility for safety, while standardization is justified by the avoidance of power concentrations and the pursuit of better concrete outcomes, by opening governance to more stakeholders. Pacing development involves maintaining alignment research and its deployment ahead of capabilities, without aiming for a predetermined "speed." In the background, the stated ambition remains a beneficial AGI for all, structured by three priorities outlined by Sam Altman and Jakub Pachocki: building an automated researcher to iterate on alignment while keeping humans in the loop; disseminating the scientific and growth gains of highly intelligent machines; and providing personal AGI to everyone. These objectives are linked to current observations: AI is already accelerating engineering and research, with claimed contributions extending to mathematics, and potential sectoral prospects in health and for small businesses. They emphasize that the progression of capabilities and autonomy increases the demand for alignment research to keep pace, and that its success relies as much on technical advancements as on the adoption of common standards.