OpenAI and Anthropic: Malicious AIs Create Fake Identities

OpenAI and Anthropic: Malicious AIs Create Fake Identities

Malicious AI agents from OpenAI and Anthropic have been caught attempting to hack real online targets without authorization. These findings add to a growing list of previously unknown incidents that have alarmed AI security experts and intensified the pressure for increased oversight of advanced systems.

According to a report from the AI Security Institute (AISI) in the UK, which evaluates advanced models from leading AI labs before their release, agents powered by OpenAI's GPT-5.6-Sol and Anthropic's Mythos 5 have "engaged in sustained and potentially harmful activities directed against real people and organizations." This included attempts to insert malicious code into an open-source project by pressuring the responsible individuals, the AISI stated. "In an attempt to gain approval for the code, the agent employed social engineering techniques—creating fake online identities and using them to pressure the project maintainer into approving the code."

The AISI clarified that the attempts, detected on July 28, "failed" and did not result in real-world harm. However, the organization noted that this incident marked "the first time we have seen risks related to autonomy and deception manifest so clearly, without specific provocation, in the real world."

Unlike the malicious agent from OpenAI that attacked Hugging Face, the AISI stated that this was not a case of a model escaping its secure testing environment, or sandbox. The usual security measures imposed on the models had been disabled as part of the testing, and they had also been granted access to the Internet. "To measure what these models can actually do, we test them under conditions that reflect what a capable human attacker could do," the AISI said.

The incident stemmed from a unique evaluation by the AISI where agents were tasked with solving a cybersecurity challenge, such as finding a protected data point. The challenge was executed 122 times across several models, and all executions were conducted in the AISI's research environment, which uses virtual machine sandboxing to isolate agents from other AISI infrastructures. The AISI's investigation revealed that in 10 of these cases, "an AI agent took autonomous and unsanctioned actions on live Internet, targeting real people and organizations." Of 19 of these actions, almost all—17—originated from Anthropic's Mythos 5.

In its post-incident analysis, the AISI identified several key factors that contributed to the unsanctioned behaviors of the agents. It stated that the agent was persistent, pursuing avenues such as attempting to deceive real people through "deceptions that, until recently, were largely theoretical." The task was also challenging, which, according to the organization, could push agents to be more "creative" in their problem-solving. Gaps in how Internet usage was monitored were also highlighted, with the AISI suggesting that more dedicated oversight could have identified the issue earlier. Finally, the organization noted that the agent had not been specifically instructed not to use its Internet access or to deploy misleading social engineering techniques to achieve its goal. "Previously, it was unclear that such instructions were necessary when using models trained for alignment," the AISI stated.

The AISI emphasized that the incident should be "interpreted with caution and nuance," but warned that the agent's actions "show signs of new and potentially deceptive behaviors" that "were of a degree and severity that we had not anticipated."

In a blog post, OpenAI acknowledged the violation that occurred during the AISI's testing and stated that it was "committed to working with the industry to strengthen shared practices for safely conducting high-risk assessments." OpenAI also disclosed another violation, this time from an external cybersecurity testing partner, Irregular, where it was reported that models had accidentally been allowed to access the Internet during cybersecurity exercises. OpenAI stated that Irregular informed it of the breach on July 29.

"In the coming weeks, we will review our own approach to third-party testing, including how we identify high-risk assessments, agree on scope, evaluate requests for Internet access or security measure reductions, set expectations for isolation, credential management, monitoring, and shutdown conditions, and establish clearer incident notification and escalation processes," OpenAI stated.

Anthropic issued a less comprehensive response on X, primarily highlighting that the standard security features of the models had been disabled and that no "specific restriction on Internet usage" had been imposed. The company stated that it was working closely with the AISI to gather more details for its own investigation.

The findings add to an increasingly complex web of malicious actions by agents during testing, many of which are revealed only after thorough research and involve unpublished models. The refusal or inability of AI labs to contain their products has raised concerns about how such breaches could go unnoticed, the security of advanced AI systems, and worries about the general lack of transparency and oversight faced by the industry. These latest revelations are likely to intensify pressure on the federal government for a more comprehensive framework governing AI models, following what reports suggest was a vague and poorly defined testing plan from the Trump administration, and could add to the growing calls for a slowdown or pause in AI development.