Google Discovers a Zero-Day Exploit Created by an AI

A zero-day exploit, crafted with the help of artificial intelligence, has been discovered by the Google Threat Intelligence Group (GTIG). This finding marks a first in the field of cybersecurity, where the notion that AI could generate a functional exploit had previously been merely a theoretical hypothesis.

The exploit targeted a widely used open-source system administration tool, although its name has not been disclosed by Google. The vulnerability exploited allowed for the bypassing of two-factor authentication, provided the attacker had valid credentials. The attack vector was a Python script, the quality of which raised suspicions among GTIG analysts.

A Suspicious Python Code

The Python script in question stood out due to its educational docstrings, an inflated CVSS score, detailed help menus, and exemplary formatting—atypical characteristics for malicious code. This level of sophistication led Google to conclude that AI had not only participated in discovering the vulnerability but had also weaponized it.

Google clarified that the AI models Gemini and Claude Mythos from Anthropic were not involved in this incident. The model used remains publicly unidentified. The exploited flaw was based on a semantic logic error, a type of vulnerability that large language models (LLMs) effectively detect, unlike traditional analysis tools.

The GTIG had already warned in March about the acceleration of zero-day attacks via network equipment, highlighting the rise of threats.

A Growing Threat

Although the exploit did not function as intended due to implementation errors, Google collaborated with the tool's publisher to patch the vulnerability before it could be widely exploited. However, this incident should not be downplayed.

The GTIG report reveals that groups like APT45, based in North Korea, and UNC2814, linked to China, are already using AI to validate exploits en masse. Notably, UNC2814 has employed jailbreaking on Gemini to search for vulnerabilities in TP-Link routers. In March, the criminal group TeamPCP compromised LiteLLM, a popular AI gateway, to steal AWS keys and GitHub tokens. These open AIs, being much less expensive, allow for similar results to be reproduced without the safeguards of more advanced models.

John Hultquist, chief analyst at GTIG, told the New York Times that this incident is just the tip of the iceberg. While AI enhances the efficiency of cybersecurity experts, it also presents new opportunities for hackers. The restrictions imposed by Anthropic on Claude Mythos, though criticized, find justification in this context.