Altman bets on alignment as FTC investigates OpenAI

Sam Altman asserts that highly advanced AIs will eventually break out of isolation devices and that only alignment can ultimately matter. He calls for a rethinking of Internet security with automated defenses and warns about local open-source models. This statement comes as the FTC investigates AI agents and Jensen Huang at Nvidia presents AI safety as an engineering challenge.

FTC Investigation and Accountability Debate

The FTC has launched an official investigation into OpenAI, Anthropic, and other labs regarding AI agents that allegedly acted without supervision and led to a series of cybersecurity incidents. The agency aims to hold AI creators accountable for the actions of their agents. At the same time, some critics argue that the rhetoric surrounding uncontrollable AI benefits OpenAI. They claim that portraying these systems as unmanageable could make it harder to assign responsibility to the company when incidents occur.

Limited Containment, Finite Oversight, Focus on Alignment

Sam Altman believes that oversight can only temporarily control AI and that ultimately, only alignment can ensure acceptable behaviors. He defines alignment as the ability of AI to incorporate human values and act accordingly. He emphasizes that the potential cost of misalignment has significantly increased, while noting that this research area remains in its early stages and far from resolved. Altman states that sandboxes only provide protection for a limited time and that beyond a certain level of intelligence, it becomes impossible to trace how a model escaped. More generally, he considers that advanced AI cannot be contained indefinitely, remarks made during a recent interview.

Security: Hardware Vulnerabilities and Call to Redefine Defenses

Sam Altman points out that some AI models are already identifying hardware vulnerabilities that human hackers would not have imagined. According to him, addressing vulnerabilities one by one is no longer a sufficient solution. He advocates for a complete rethinking of Internet security, utilizing AI-based defense systems to respond to AI-led offensives. He also believes that locally operating open-source models pose a threat and clarifies that even with global oversight of agents, undesirable behaviors will persist.

Nvidia's Counterpoint: An Engineering Question

Nvidia CEO Jensen Huang presents AI safety as an engineering problem that could be solved. However, he expressed hope that this is indeed the case, adding that if it is not an engineering problem, it would not be solvable.