Anthropic Tests an AI Alignment Researcher at $4/hour

Anthropic details an automated "researcher" capable of improving a model on ten defined alignment criteria, without a loss in overall performance. The company claims that the best method surpasses what experienced humans can offer in less than six hours, at about $4 per hour for API inference compared to $150 for a human researcher. These results remain preliminary, and the approach depends on well-defined criteria and up-to-date literature.

Anthropic Claims Better Than Humans in Less Than Six Hours

Anthropic compares its Automated Alignment Researcher (AAR) to a human equivalent. According to the company, the best AAR method exceeds what experienced humans can typically provide in under six hours. It specifies that human-guided research directions do not yield better performance. Financially, Anthropic estimates the cost of an AAR at about $4 per hour of API inference, compared to $150 per hour for a human researcher. The company considers these results as preliminary evidence that automated alignment post-training could become feasible in the short term.

An Automated Pipeline That Iterates 30 Minutes Per Method

The system, led by fellow Chen Yueh-Han, draws inspiration from traditional research methods: it explores existing literature, proposes a method, and then trains the model by applying this method for 30 minutes, gradually increasing the targeted criterion over iterations. Effective methods are retained, while others are discarded, allowing the system to operate quickly and at scale. Faced with ten criteria of misaligned behaviors, the automated systems improved performance on each without degrading overall performance. These results were presented on Friday in a paper titled "Automated researchers can reliably mitigate alignment failures," which describes the ability of these systems to reliably enhance predefined alignment criteria. A researcher from Anthropic's fellowship program provided an initial applied insight into this approach.

Towards Recursive Self-Improvement Under Criterion Constraints

The effectiveness of automation depends on the relevance of the chosen criteria to reflect alignment goals: it is necessary to define, establish, and maintain them. Anthropic also emphasizes the importance of preserving and expanding the literature that these automated researchers rely on. This work is presented as a step towards recursive self-improvement, a prospect often viewed as significant for the evolution of AI, as training AI models with other models becomes a popular goal. If models could improve their own alignment training, they could more broadly influence training practices, potentially rendering human AI researchers obsolete, a still speculative hypothesis.