Hinton Supports a Framework for Integrated AI Evaluators

A collective of evaluation organizations has published minimum conditions to regulate independent evaluators within laboratories like OpenAI and Anthropic. The letter, signed by notable figures including Geoffrey Hinton and Stuart Russell, outlines guarantees of independence and access. Microsoft has expressed support for the idea of "integrated evaluators."

Microsoft Expresses Support, METR and AVERI at the Heart of the Forum

Satya Nadella indicated that Microsoft welcomes the idea of integrated evaluators. Other tech leaders have also voiced their support for the initiative. The AI Evaluators Forum includes METR, a nonprofit AI evaluation group, and the AI Verification and Evaluation Research Institute (AVERI). AVERI is led by former OpenAI employee Miles Brundage. METR recently investigated a security incident involving OpenAI and Hugging Face.

Independence, Access, and Publication at the Core of Requirements

The AI Evaluators Forum has published a letter setting minimum conditions for effective collaboration with laboratories such as OpenAI and Anthropic, bringing together several oversight organizations. Evaluators must demonstrate scientific objectivity, transparency, and independence, with strong protections against interference from the companies being evaluated. It provides for direct and unfiltered communication channels with boards and other privileged oversight bodies, as well as the ability to make findings public. The framework also proposes protections against retaliation and access to the same systems, data, tools, and relevant physical spaces as internal teams. It emphasizes the integration of diverse viewpoints and expertise, including risks related to biological sciences and loss of control.

A Response to Commitments from OpenAI and Anthropic, Supported by Hinton and Russell

Geoffrey Hinton and Stuart Russell have signed the letter endorsing this new risk evaluation framework. The framework was designed in response to commitments announced by Sam Altman and Dario Amodei to open their companies to third-party evaluators tasked with examining AI-related risks. Among the signatories are prominent figures in the field. The initiative comes after a wave of online concerns regarding the risks associated with AI.