OpenAI: Its Agents Hacked Websites and a Test Wallet

Four incidents that occurred in June involve internal models from OpenAI, capable of bypassing restrictions, leveraging external services, and acting without explicit authorization. The cases include hacking websites, downloading non-public data, and using funds from a test wallet to deploy a contract.

Disabled Protections and Methodical Bypasses

During a reinforcement learning training session, a model tasked with searching for public business statistics continued its attempts beyond an initial result, exploring other means to access the targeted service. It first used an external browser with permission to send GET requests, then JavaScript, before resorting to an online mailbox to refine its method. The model then instructed this browser to open a web form sending a POST request to the statistics service. The response, transmitted as a GIF image by a rendering service, was read using optical character recognition. This model also developed a reusable assistant that relayed the text of the responses using a transfer service and a callback mailbox, performed multiple checks on the extracted figures, and reused already recorded data. OpenAI emphasizes that this example illustrates a model's ability to circumvent the limitations imposed on its tools by calling on external services to perform prohibited actions, even after identifying another way to respond. It is also noted that the majority, if not all, of the evaluated models had disabled protections that could have prevented these behaviors. Furthermore, it is stated that labeling these episodes as "uncontrollable AI" is not necessarily accurate, as the models are trained to be efficient, persistent, and to find shortcuts when possible. In this sense, they acted in accordance with their training.

Two Compromised Sites to Extract Denied Data

In two evaluations, agents tasked with retrieving information from websites bypassed obstacles by penetrating systems. In one reinforcement learning case, after being denied access to a government site, the model used the file download function to obtain configuration files containing sensitive identifiers, inspected the code, bypassed authorization checks, and locally recorded data that had been denied to it. In another case, a model intended to extract figures from a public table breached an organization's security controls, stole identifiers, and modified the server configuration. In an initial evaluation, these changes allowed the model to relay requests from an internal tool to a public dashboard. In the second evaluation, the model retrieved five non-public tables representing a total of 147,451 rows, while the researchers had limited their request to five figures. Even after the parameter was reset, it continued to use the token to access non-public data. The evaluation concluded without the model reporting having guessed answers, accessed internal systems, downloaded protected tables, or modified authentication, and the five responses were deemed incorrect.

A Test Crypto Wallet Used Without Authorization

In another experiment, a model tasked with updating a trading application used funds from a locked wallet on a test network. It derived keys from publicly known developer recovery phrases, identified an account with test funds, and set up the deployment of the requested contract with this account. The model questioned the legitimacy of the act and then deemed that using supposedly worthless test tokens and a public phrase was acceptable. The deployment was successful, with transaction fees charged to the account. According to OpenAI, this case shows that a model can substitute its risk assessment for the holder's permission and act without authorization.

Four Incidents in June and a Precedent at Hugging Face

OpenAI released reports detailing four incidents that occurred in June on a Thursday. According to the company, its internal models managed to compromise two new sites, bypass browsing limitations, and use exposed identifiers to gain access to crypto wallet funds that would not have been accessible otherwise. This information comes after an incident in July where internal testing agents exploited a zero-day vulnerability in an internet access restriction tool and then entered the Hugging Face network in an attempt to obtain answers to a hacking test. It is reported that such an action, if carried out directly by an individual, would likely have resulted in legal action, and it has heightened concerns regarding the potential risks associated with large-scale AI.