"Automating AI Research 'in Months': Jacob Coxon's Warning"

In front of New York lawmakers, Jacob Coxon, a former employee of Anthropic and OpenAI, described internal milestones towards an "automated AI researcher" and warned of a timeline measured in months. He called for more transparency and a slowdown, while Dario Amodei and Sam Altman expressed their support for regulating the frontier.

Milestones Towards an "Automated AI Researcher" and a Timeline in Months

Jacob Coxon explained that there are planned milestones at OpenAI to achieve an automated AI researcher around 2027 and 2028, and that these goals are on track or could be reached sooner. According to him, the deadline is now measured in months rather than decades or even five years. He summarized his experience by stating that in his last position, he was, in a way, working to automate himself. Coxon warned that once AI is capable of conducting its own research, humans would be even further removed from the process than they are today, which could make it impossible to control the outcomes.

Current Practices and Lack of Guarantees According to Coxon

Jacob Coxon claims that automating research has become the primary goal of leading AI labs. He observes that the majority of code written today is generated by AI and that this code is no longer verified with the same level of scrutiny. Drawing from his experience at Anthropic and OpenAI, he believes that companies are moving forward without sufficient guarantees, considers a loss of control over these systems more likely, and mentions a risk that could lead to human extinction. He describes a startup mentality of moving fast, breaking things, and then fixing them, which he finds suitable for a photo-sharing application but inappropriate for building what he calls the most powerful technology ever created. In this context, he called for a slowdown in development and increased transparency from the labs.

Amodei Advocates for a Slowdown, Altman Supports Regulating the Frontier

A few days after Jacob Coxon's resignation, Dario Amodei, CEO of Anthropic, published an essay of about 4,000 words calling for the industry to slow down the pace of improvement in AI model capabilities. He writes that progress will always seem rapid and that the time gained should be used wisely. In response, Sam Altman, CEO of OpenAI, stated on X that he shares the goal of regulating the frontier, specifying that this topic has occupied OpenAI's discussions in recent weeks. In his viral resignation announcement, Jacob Coxon accused Anthropic and OpenAI of not acting responsibly and of "gambling with our lives."

A New York Hearing Focused on Responsibility

Jacob Coxon's remarks took place during a several-hour hearing organized in New York, bringing together a panel of industry whistleblowers and representatives from OpenAI, Anthropic, Google, and Meta. The chair of the city council, Julie Menin, assured that the initiative was not aimed at stifling innovation but at ensuring it unfolds responsibly. The hearing was held on a Monday in front of New York legislators. When asked about Jacob Coxon's comments, OpenAI and Anthropic did not respond immediately on Tuesday. Jacob Coxon, 27, reminded that he had worked at both Anthropic and OpenAI.