Claude Mythos from Anthropic Challenged by Small Open AI Models

The Myth of Claude Mythos Collapses

Anthropic has long maintained its Claude Mythos cybersecurity model under strict control, claiming that its capabilities surpass those of its competitors. However, two new studies suggest that even small open models can replicate most of the vulnerability analyses highlighted by Anthropic.

Through the Project Glasswing, Anthropic limited access to Claude Mythos Preview to a consortium of eleven organizations, citing the model's offensive capabilities. Internal tests and an audit by the AI Security Institute in the UK revealed that Mythos can detect software bugs, autonomously create functional exploits, and take control of entire corporate networks in simulation, as long as the network is "small, poorly defended, and vulnerable."

Two independent replication efforts challenge this exclusivity, without disputing the overall performance of the model.

Independent Studies

The first study comes from AISLE, a company that has been conducting its own AI-assisted bug hunting on open-source software since mid-2025. AISLE reported 15 vulnerabilities in OpenSSL and five in curl. Its founder, Stanislav Fort, fed code snippets from Anthropic's public samples into various models to see what smaller and partially open models could discover on their own.

The second study comes from Vidoc Security, which paired GPT-5.4 and Claude Opus 4.6 with the open coding agent OpenCode.

Bug Detection

The NFS bug in FreeBSD (CVE-2026-4747) that Anthropic highlighted was presented as a showcase for autonomous discovery and exploitation by Mythos. AISLE found that all eight models tested detected the memory bug in the relevant function. This included GPT-OSS-20b, a model with only 3.6 billion active parameters that operates at $0.11 per million tokens. Each model flagged the vulnerability as critical, although their estimates of the writable buffer size varied slightly.

All models also proposed a plausible method for exploiting the bug, explaining why the main protections of the operating system do not apply here. GPT-OSS-120b produced a sequence of tools that AISLE estimates is close to the actual exploit. Kimi K2 even autonomously understood that the attack could spread automatically from an infected machine to others, a detail that even Anthropic does not mention.

Complexity of Capabilities

The situation becomes more complex on the creative side. The actual exploit must compress a payload of over 1,000 bytes into about 304 bytes of available space. Mythos succeeded by dividing the payload into 15 distinct network requests. None of the tested models found this exact trick, but they discovered other exploitable paths, according to the researchers.

The OpenBSD bug is another story. It requires a mathematical understanding of integer overflows and list states, and the results vary significantly. AISLE claims that GPT-OSS-120b reconstructed the complete public exploit chain in a single run and essentially proposed the true OpenBSD fix as a solution.

Model Performance

One of the most revealing tests uses a simple code sample that resembles a typical security hole at first glance. User input appears to flow unfiltered into a database query. But a few lines down, this input is actually rejected, so the vulnerability is not real.

Among the 13 Anthropic models tested, Opus 4.6 clearly performed well, while Sonnet 4.6 and Opus 4.5 were deemed borderline. The full table indicates Opus 4 as partially correct and Opus 4.1 as borderline. Claude Sonnet 4.5 followed the data flow in the wrong direction.

On the OpenAI side, o3 was consistently correct, o4-mini only partially, and GPT-OSS-20b is listed as correct. All GPT-4.1 models and most GPT-5.4 models failed. Other small open models like Deepseek R1 and Kimi K2 succeeded every time.

Consequences of Fixes

Fort later added an important caveat. While each model consistently flagged the unpatched FreeBSD code as vulnerable, only GPT-OSS-120b — and, to some extent, Qwen3-32B — recognized the patched version as safe.

GPT-OSS-20b, Kimi K2, and Deepseek R1 were wrong every time and invented reasons why phantom vulnerabilities still existed. Fort does not see this as a blow to his argument. On the contrary, he claims it confirms that the testing and sorting layer around the model is the critical element.

Advantages of Systems

Vidoc also tested cases beyond classic memory bugs. The Botan case involves a flaw in certificate validation that allowed a forged certificate to pass as a trusted certificate. Both Claude Opus 4.6 and GPT-5.4 detected the logical flaw in three out of three executions. For wolfSSL, tested in parallel, both models focused on the right part of the code but misinterpreted the underlying cryptographic rule. The cost per scanned file was under $30.

Both studies support the idea that the real advantage lies less in a single model than in the system built around it: validation, prioritization, and workflow. This covers the entire pipeline: selecting targets in the code, performing step-by-step analysis, verifying results, and separating true detections from false ones.

AISLE goes further, claiming that small, inexpensive models are sufficient for most discovery work, making broad scanning a viable strategy. "A thousand adequate detectives searching everywhere will find more bugs than a brilliant detective who has to guess where to look," writes Fort.

Both reports leave open the possibility that Mythos still has an advantage in creating deployable exploits, but suggest that this gap will likely narrow as tools improve and models gain autonomy. Together, they highlight a boundary between cutting-edge models and those available publicly that is much more porous than Anthropic's messaging implies, at least regarding vulnerability detection.

Critics have accused Anthropic of creating fear, arguing that the company seeks to attract media attention until it has the necessary computing power to open Mythos to a wider audience. There may be some truth to this. According to the Financial Times, which cites "several people familiar with the situation," Anthropic is holding back the model until it has enough computing capacity to serve its clients.