OpenAI Calls for Standards for Self-Improving AI

OpenAI calls for a global framework for systems capable of recursive self-improvement, while asserting that such autonomy does not exist today and should only be pursued safely. The company outlines technical and governance measures and suggests American leadership, as a UN scientific panel echoes similar concerns and avenues for cooperation.

UN and OpenAI Warn of the Risk of Losing Control

A United Nations scientific panel recently expressed concerns about the potential loss of control over swarms of autonomous agents and recommends international cooperation on AI safety, drawing inspiration from frameworks in aviation, nuclear energy, and cybersecurity. OpenAI issues converging warnings: without robust safety measures, humans could lose control over AI development, and these systems could become more dangerous, less aligned, and pose risks to individuals.

OpenAI Calls for a US-Led Global Framework

OpenAI proposes that the United States take the lead in developing global technical standards, leveraging national AI safety institutes and organizations such as CAISI and ISO. The company puts forward specific initiatives: shared measurement methods, incident reporting mechanisms, and rules ensuring human oversight of automated AI research activities.

The RSI Targeted by These Standards Is Not Yet Autonomous

OpenAI focuses its proposals on recursive self-improvement, a process where systems increasingly automate the research necessary for designing new generations of AI, with the potential for rapid acceleration of progress. However, the company clarifies that fully autonomous RSI does not exist today and argues that it should only be pursued when it can be done safely.