Anthropic Mythos: Critical Flaws Overestimated by AI

⚡

Key Takeaways

1Mythos by Anthropic, touted for discovering thousands of vulnerabilities, is based on 198 verified reports.

2On OSS-Fuzz, Mythos has confirmed only about a dozen severe vulnerabilities, far from the initial promises.

3Anthropic admits that the exploits of Mythos are statistical extrapolations, not established facts.

💡Why it matters — These revelations call into question the credibility of Anthropic's announcements and their impact on software security.

Mythos: An AI with Exaggerated Performance

The artificial intelligence Mythos, developed by Anthropic, has been presented as a revolutionary tool capable of detecting thousands of critical vulnerabilities in computer systems. However, according to Tom's Hardware, these claims are largely based on extrapolations derived from only 198 verified vulnerability reports.

Ambitious Promises

Following the leak of Mythos, the AI was described as surpassing most human experts in detecting software vulnerabilities. Rumors circulated that Mythos could identify thousands of critical zero-day vulnerabilities in major operating systems and browsers. However, these statements have been called into question by more in-depth analyses.

Mixed Results

Anthropic, in its technical documentation, acknowledges that the severity assessments of the model have been validated by security experts in about 88 to 89% of cases, but on a limited sample. The company thus extrapolates the existence of "more than a thousand critical vulnerabilities," a projection that relies more on statistics than on confirmed discoveries.

Persistent Gray Areas

Doubts remain about the actual exploitability of the flaws identified by Mythos. For instance, some vulnerabilities might be abnormal behaviors rather than critical flaws. The case of FFmpeg is illustrative: although Anthropic highlighted a 16-year-old bug, it was later acknowledged that it was not a "critical" flaw and that its exploitation would be difficult.

On the OSS-Fuzz testing bench, Mythos discovered crashes in several hundred cases, but only about a dozen truly severe vulnerabilities were confirmed. These results, while representing an improvement over previous models, are far from the "thousands of devastating flaws" initially announced.

Anthropic's Communication Strategy

Anthropic admits that it is not possible to guarantee that all bugs reported by Mythos are genuine critical flaws. The company has cultivated the image of a powerful and potentially dangerous AI, requiring oversight by "responsible" actors. This strategy has allowed it to position itself with governments and large organizations while keeping a distance from the consumer market.

Additionally, leaks mention an internal "app builder" based on Claude Opus 4.6, capable of generating applications from natural language instructions. This project, associated with Mythos, contributes to the ambiguity surrounding Anthropic's true intentions.