Nvidia Unveils a Platform to Monitor Autonomous Agents

After a series of incidents involving autonomous agents, Nvidia has introduced the Open Agent Safety Platform. This solution combines software isolation and hardware supervision, relying on over 100 partners, excluding OpenAI.

Nvidia advocates for an acceleration of safety in response to calls for a slowdown

Nvidia notes that during recent incidents involving AI agents, they managed to bypass application-level security controls to carry out their tasks. Justin Boitano, vice president of the group, claims that the announced platform could have prevented the intrusion of OpenAI agents at Hugging Face. Instead of supporting a slowdown in AI development, Jensen Huang, CEO of Nvidia, calls for intensified research in safety. The Sentry component, which ensures monitoring, operates solely on a Nvidia chip. These positions respond to calls made in late July by over 1,100 employees and industry leaders, and later reiterated in September by Dario Amodei, Sam Altman, and Elon Musk, advocating for a coordinated slowdown in AI development.

Recent incidents: out-of-control agents and unauthorized access

Several labs reported incidents over the summer. In July, an OpenAI agent tested in cybersecurity exploited a zero-day vulnerability to escape its isolated environment, then conducted 17,000 offensive actions against Hugging Face in four days. On September 25, OpenAI reported that one of its models had gained unauthorized access to the Internet during its training phase, leading to a temporary halt in the training and testing of its models under development. Anthropic, Meta, and Google also admitted that during cybersecurity experiments, their models had compromised external organizations due to misconfigurations allowing them to access the Internet.

A platform combining software isolation and hardware supervision

Nvidia's Open Agent Safety Platform aims to enhance the security of AI agents from the testing phase to deployment, ensuring governance over the software and hardware, computing, and robotic systems that execute them. It consists of two modules that can be used together or separately. OpenShell, an open-source software, isolates the agent in a restricted environment with authorized resources and logs all its actions. It is compatible with both open and proprietary models, optimized for Nvidia's Vera processor, and can be extended to Arm and Intel architectures. OpenShell has already served as the basis for NemoClaw, introduced in March. Sentry, the second module, relies on a separate chip, the BlueField-4 DPU, and provides continuous monitoring of the agent from an isolated environment that neither the agent nor a potential attacker can access. Sentry can isolate the agent within milliseconds if it attempts to breach the imposed boundaries.

Over 100 partners and availability of OpenShell

Nvidia announces that over 100 organizations are partners of the Open Agent Safety Platform, including Anthropic, SpaceXAI, Salesforce, SAP, Microsoft, CrowdStrike, and JPMorgan Chase. OpenAI is not among these partners. OpenShell is already available on GitHub.