Hugging Face Under Attack: An AI Agent Highlights Cybersecurity

Hugging Face Under Attack: An AI Agent Highlights Cybersecurity

An autonomous AI agent has conducted a full-scale intrusion into the infrastructure of Hugging Face. To analyze the attack, the platform faced rejection from commercial large models, before finding salvation in Chinese open source. Hugging Face provides us with a unique case of AI defense against autonomous agents.

Scenarios of attacks entirely driven by artificial intelligences had previously circulated in cybersecurity conferences, somewhere between anticipation and marketing argument. Hugging Face has just provided a first large-scale documented case. In a post published on July 16 on its blog, the model hosting platform recounts having suffered an intrusion carried out end-to-end by a system of autonomous AI agents. And the way it defended itself speaks volumes about the attack itself.

A Trapped Dataset, A Weekend Stroll Through Clusters

The entry point is nothing exotic for those familiar with AI platforms: the data processing pipeline. A malicious dataset exploited two code execution paths (a remote data loader and an injection into a configuration model) to gain foothold on a processing machine. From there, the attacker escalated to the node level, harvested cloud and Kubernetes credentials, and navigated from cluster to cluster throughout an entire weekend. The whole operation was orchestrated by a system of agents executing thousands of actions across a swarm of ephemeral sandboxes, with a command infrastructure that moved autonomously across public services. The language model that powered the attacker remains unknown to this day.

The company's assessment aims to be reassuring: unauthorized access to a limited set of internal datasets and several service credentials. However, there was no trace of alteration to models, datasets, or public Spaces, and a verified healthy software chain. The evaluation of any potential impact on customer or partner data is still ongoing. On the remediation side, vulnerabilities have been patched, nodes rebuilt, secrets massively renewed, and the on-call protocol revised so that a critical signal can wake someone within minutes, even on weekends (the experience has evidently served as a wake-up call).

When AI Safeguards Block the Good Guys

It is in the analysis of the attack that the story takes an unexpected turn. To dissect the attacker's operation log, comprising over 17,000 events, Hugging Face deployed its own analysis agents. Reconstructing the timeline, extracting indicators of compromise, sorting between real actions and decoys: all of this took a few hours, whereas a human team would have counted in days. A logical reflex, until it came time to choose the model. The commercial large models queried via their APIs refused to process the requests: submitting real attack commands and payloads triggered their safeguards, unable to distinguish a defender from an attacker. It’s a bit like a security guard refusing to let a locksmith in because he carries burglary tools. The company does not name any provider but emphasizes the asymmetry: the attacker was not bound by any usage policy.

The workaround came from GLM 5.2, an open-weight Chinese model published by Z.ai, executed on in-house infrastructure. This had a double benefit, as neither the attack data nor the credentials they referenced left the environment. The lesson is valuable for all security teams: having a self-hosted model, validated before the incident, rather than discovering the lock during a crisis. This is yet another plea for open AI, coming from this company founded by three French individuals that has become a heavyweight in open-source AI. Not without a certain sense of timing: the platform, which launched its own free AI agent last year, has just received a visit from a similar agent that has crossed to the dark side.

The crisis exercise is nothing new for Hugging Face, which had already endured three security incidents in three months at the beginning of 2024, including a token theft. The difference this time lies in the nature of the attacker, which changes the scale of the problem: a patient, multi-step campaign conducted at machine speed.

In the meantime, the company's advice applies to all accounts: rotate your access tokens and take a look at your recent activity. The attacker may have been a machine, but the precautions remain very human.