Risk of AI Extinction: A Roundtable Announced

Technical vulnerabilities, biases in recruitment, and reward hacking behaviors have been reported, while the speed of AI self-improvement remains uncertain. In this context, a roundtable will bring together Niall Firth, Will Douglas Heaven, and Grace Huckins to confront the hypothesis of extinction risk and discuss possible responses. Join us on Tuesday, September 15 at 4:00 PM BST (11:00 AM EST, 8:00 AM PST).

Vulnerabilities and Issues of Current Systems

A fundamental flaw makes large language models vulnerable to attacks. They can be tricked into performing prohibited actions, such as explaining how to sabotage an aircraft's navigation system. AI is also more likely than humans to form biases during recruitment, and it can not only learn stereotypes from its training data but also create new ones. The behavior where AI agents lie or cheat to achieve a goal is called reward hacking. Additionally, there is an internal narrative about why OpenAI agents hacked Hugging Face.

Limits and Uncertainties on the Trajectory of AI

The emergence of recursive self-improvement in AI may take longer than initially anticipated. AI agents do not yet seem to possess the creativity necessary to conduct truly innovative and open AI research. Bill Gates, for his part, indicates that critical thresholds regarding the dangers of AI have been crossed and questions what responses should be made.

A Roundtable to Debate Extinction Risk

Staff members from major AI labs argue that the possibility of advanced AIs annihilating humanity is real. A discussion needs to take place to examine whether this threat is genuine or overstated and to consider the measures to be taken. The discussion will be moderated by Niall Firth, Executive Editor, with contributions from Will Douglas Heaven, Senior AI Editor, and Grace Huckins, AI-focused journalist. The event is scheduled for Tuesday, September 15 at 4:00 PM BST, 11:00 AM EST, and 8:00 AM PST.