OpenAI and the Pentagon: An AI Agreement That Raises Concerns

⚡

Key Takeaways

1OpenAI has signed an agreement with the Pentagon, incorporating security principles against mass surveillance and autonomous weapons.

2Anthropic had its contract terminated by the Pentagon for refusing to modify its agreement to include "any legal use" of its technology.

3Doubts are emerging about the reality of the concessions obtained by OpenAI, as its agreement appears to be less restrictive than that of Anthropic.

💡Why it matters — This agreement raises questions about the influence of tech companies on defense policies and the transparency of commitments made.

OpenAI and the Pentagon: A Controversial Agreement

In a series of recent events, OpenAI has reached an agreement with the Pentagon for the deployment of classified artificial intelligence. Sam Altman, CEO of OpenAI, stated that this agreement incorporates crucial security principles, including a ban on mass surveillance and autonomous weapons. Altman emphasized that human involvement in high-stakes automated decisions is essential, and these principles have been integrated into the contract with the Department of Defense (DoD).

Anthropic and the Pentagon: A Contract Terminated

Meanwhile, Anthropic, another AI technology company, has seen its contract with the Pentagon terminated. The reason cited is Anthropic's refusal to amend its agreement to allow for "any legal use" of its technology, including for mass surveillance and autonomous weapons. The Pentagon also threatened to designate Anthropic as a "supply chain risk," a measure typically reserved for companies associated with foreign adversaries.

Sam Altman's Statements

On Friday morning, a report from the Wall Street Journal by Hagey revealed that Altman had sent a memo to OpenAI staff, stating that he would establish the same "red lines" as Anthropic. Altman wrote: "We have long believed that AI should not be used for mass surveillance or lethal autonomous weapons, and that humans should remain involved in high-stakes automated decisions. These are our main red lines."

Agreement with the Pentagon

On Friday evening, Altman announced on X that OpenAI had reached an agreement with the Pentagon, asserting that the same red lines were now integrated into the contract. Altman stated: "Two of our most important security principles are the prohibition of domestic mass surveillance and human accountability for the use of force, including for autonomous weapon systems. The DoD agrees with these principles, reflects them in law and policy, and we have incorporated them into our agreement."

Reactions and Skepticism

Altman's claim was immediately scrutinized. Doubts emerged about how quickly OpenAI had secured concessions from the Pentagon, while Anthropic had failed. Was it possible that the same Pentagon officials who opposed the idea of a private company attempting to exert control over the military were now making an exception for OpenAI? Was the public now, like Mira Murati and Ilya Sutskever before them, caught in the familiar trap of Altman starting by telling them what they wanted to hear?

Online Reactions

Notably, few seemed willing to give Altman the benefit of the doubt. The most popular post on the ChatGPT subreddit over the past week was titled "You are now training a war machine. Let's see the proof of cancellation"; it received over 32,000 upvotes. Similar posts in this forum and the OpenAI subreddit also garnered tens of thousands of upvotes; the company was also extensively criticized on Hacker News.

Additional Revelations

As the weekend progressed, further reports suggested that the instinctive cynicism triggered by OpenAI's agreement was justified. In The Verge, Hayden Field reported that contrary to OpenAI's public statements — and in line with the military's framework regarding its requirements — the company's agreement with the Pentagon includes fewer restrictions than the one Anthropic had proposed. She writes:

A source familiar with the Pentagon's negotiations with AI companies confirmed that OpenAI's agreement is much more flexible than the one Anthropic was trying to secure, largely due to three words: "any legal use."
During negotiations, the individual stated that the Pentagon did not want to relent on its desire to collect and analyze massive amounts of data on Americans.
When examining the terms of OpenAI line by line, every aspect boils down to: if it is technically legal, then the U.S. military can use OpenAI's technology to achieve it.

Implications for Surveillance

And over the past few decades, the U.S. government has broadened the definition of "technically legal" to encompass mass surveillance programs — and more. OpenAI may be able to partially block the military's efforts to conduct domestic surveillance by building classifiers and implementing other model-level security measures, as it has stated it would. Yet, it is essential that these commitments are closely monitored to ensure compliance.