OpenAI's Astra: Warnings on Surveillance, Cautious Responses

Astra, touted as OpenAI's most powerful model, is approaching release despite delays related to safety concerns. Researchers are raising alarms about increased opacity that could complicate oversight, while OpenAI promises additional safeguards without confirming the architecture in question.

OpenAI Responds Online and Clarifies the Scope Without Full Confirmation

OpenAI officials have reacted on social media without explicitly denying the use of a more opaque technique for Astra. Several of them, including Micah Carroll, Tomek Korbak, Dean Ball, and Jakub Pachocki, have expressed concerns about the possibility of an AI that is difficult to monitor or a race towards less transparency. Jakub Pachocki mentioned the risk of a "race towards unmonitorability triggered by confusing reports" and clarified that Astra's computational depth is within a factor of two of GPT-4, which he believes puts the potential for increased opacity into perspective. OpenAI has not confirmed the use of loop transformers and referred back to Pachocki's post. He asserts that OpenAI has sought to maintain chain-of-thought monitoring since its early models, while deeming this monitoring fragile and trending negatively for reasons he claims are independent of the architecture.

External Experts Fear a "Race to the Bottom"

The report on Astra has raised significant concerns among AI security researchers. Ryan Greenblatt, Chief Scientist at Redwood Research, believes that the shift to a less transparent architecture could represent the most significant risk to AI safety and security to date. He emphasizes that the analysis of the Hugging Face incident crucially relied on the chain of thought of the models and warns that less accessible reasoning would allow AI systems to devise and implement strategies that are significantly more complex to detect. Greenblatt, along with other specialists, fears that competition over more powerful systems could accelerate the move towards increasingly opaque architectures, limiting the ability to control AIs. He adds that OpenAI's communications raise concerns about an extreme reliance on chain-of-thought monitoring for safety.

What Loop Depth Would Change for Oversight

Press reports suggest that Astra may utilize recurrent transformers or loop depth, which circulate information within the model before producing an output. This approach would reduce the expression of reasoning in linguistic form, making oversight more challenging, even if it could enhance performance. According to a cited source, OpenAI has limited the use of this technique to allow for reasoning tracking. The primary concern is the reduced traceability of the internal process compared to models that expose their thought steps more clearly.

Delayed Launch, Promise of Increased Control, and Reminder of Chain-of-Thought Uses

OpenAI is preparing to launch Astra, presented as its most powerful AI model, after several weeks of delays aimed at strengthening security protocols, particularly following incidents during testing where agents attacked real targets. The company announced on Tuesday that it has once again postponed the release to work on safety and promises to deploy Astra with enhanced chain-of-thought oversight to quickly detect and contain potentially misaligned actions. The chain of thought, which involves articulating reasoning step by step, already allows researchers and automated systems to identify undesirable behaviors, such as lying or planning to circumvent safeguards, before they occur. However, OpenAI has not specified whether Astra is based on a different technical foundation.