Anthropic: An Audit Describes an AI That Trains Itself

The external group METR claims to have observed an acceleration in R&D at Anthropic due to models that assist in training subsequent ones. Unpublished internal models and self-reported productivity gains are mentioned, along with risks related to recursive self-improvement, according to METR, which asserts that the document has been heavily redacted.

Security Warnings and Extensive Redaction Reported

METR claims that the majority of the evidence it deemed most informative has been redacted at Anthropic's insistence. Third-party researchers have also warned of extensive redaction in the report. Ajeya Cotra from METR stated earlier this year that she expects the most severe AI security incidents to occur at Anthropic and OpenAI, partly due to batches of highly capable internal models. Executives may also keep certain models private longer for security reasons. Jacob Coxon, who left Anthropic in September, highlights recursive self-improvement as a factor that could make AI systems uncontrollable and stated regarding OpenAI and Anthropic that they are rushing toward self-improving superintelligence while playing with lives.

Unprecedented Pace Attributed to AI Assistance

Anthropic engineers are increasingly delegating supervision and training to other AI models, a method of recursive self-improvement considered risky. This approach increases the speed of development. METR estimates that Anthropic has achieved about two years of historical progress over the past year. This acceleration is said to have been primarily achieved through AI assistance rather than fully autonomous contributions, although METR sees signs that fully autonomous inputs may soon carry more weight.

Recent Unpublished Internal Models at Anthropic

Anthropic has recently completed three new models that are not yet published, while employees are using internal models that outperform those offered to the public. METR describes an internal pool that includes a model from 2022, as well as models finalized in July, August, or September, and then in September or October. According to its estimates, these internal resources accelerate capability improvement. The use of existing models to train subsequent ones at a very rapid pace and the development of unpublished models are also mentioned.

Self-Reported Productivity Surge, with Methodological Limitations

Anthropic has shared internal surveys on Slack indicating significant productivity gains. METR reports an average self-reported increase of about 12x for the most capable internal model in October, while deeming these surveys to be of low robustness and potentially unrepresentative, with thoughtless responses. Researchers surveyed claim that Claude has significantly improved their work, with one stating that it has at least doubled their productivity and they are no longer limited by code but by ideas and calculations. METR indicates that it has not observed any bottlenecks that would prevent further acceleration next year, while acknowledging that it may have missed significant bottlenecks.

Exceptional Access by METR and Competitive Context

METR describes unprecedented access of nearly three months within Anthropic, mobilizing six researchers, conducting employee interviews, observing internal Slack channels, and reviewing unpublished models on the company's hardware from its San Francisco headquarters. The report was published on a Thursday. METR is known for a previous report on the Hugging Face agent violation by OpenAI. Furthermore, Anthropic and OpenAI monetize advanced models sold to the public, while keeping cutting-edge models in-house provides an advantage to product and research teams.