Anthropic Goes Public, Coxon Calls for End to RSI

Anthropic went public on Thursday, while its former researcher Jacob Coxon called for a halt to recursive self-improvement at a rally in San Francisco. The stock operation was valued at $1.7 trillion. This method involves entrusting advanced AIs with training the next generations.
Competition Worries Oliver Habryka
Oliver Habryka, the leader of the company that operates the LessWrong forum, welcomes the legal obligations for information disclosure that Anthropic is now subject to. However, he fears that the need to maintain its lead may push the company to resort to RSI. In discussions with investors, Anthropic has precisely made this lead a competitive argument.
The capabilities of AI in biology particularly concern Habryka. He believes their progress has already been rapid in recent years and fears a much stronger acceleration with recursive self-improvement, a prospect he describes as terrifying.
Coxon Calls for a Ban on RSI
For Coxon, Anthropic should seek government intervention and collaborate with other companies to completely ban RSI. He believes that companies are now responsible for the issue and that, by 2027, only a few sites worldwide could practice this method, including Anthropic's offices. He and Daniel Kokotajlo, a former OpenAI researcher, claim to be privately gathering the concerns of AI researchers. Congressional representatives have begun proposing restrictions on RSI, which Kokotajlo compares to the actions of a "comic book supervillain."
Le brief IA que les pros lisent chaque soir
Les 7 actus IA du jour, décryptées en 5 min. Gratuit.
Inclus dès l'inscription : notre sélection des meilleurs guides & comparatifs IA.
Choisissez votre rythme
Gratuit · Pas de spam · Désabonnement en 1 clic
At Anthropic, AI is already accelerating research and development activities, according to a recent independent assessment by the AI safety organization METR. However, the company warned in its IPO prospectus that it does not consider the complete design of its next generation by AI without supervision to be safe. It is considering modifying its policy to maintain human oversight in the development of the most advanced models. Anthropic did not respond to a request for comment.
Stop AI Blocks Office Entrance
In front of Anthropic's headquarters, dozens of participants, including AI safety advocates, held up signs and chanted slogans at the rally dubbed "Ban RSI." They listened to Coxon, Kokotajlo, and Rob Miles, a YouTube content creator. Later, Stop AI activists blocked the entrance to the offices. Coxon had left Anthropic in September; the concerns he expressed then had fueled fears about AI on a national scale. On Thursday, he stated that investments mattered little to him and called for an end to RSI.
David Krueger, who leads the nonprofit organization Evitable, praised the mobilization but believes it only partially reaches the employees present in the offices. According to him, they deeply adhere to their company's beliefs. He also thinks that a more informed public about RSI would generally find it "insane."
Brief IA — L'actualité IA en français
L'essentiel de l'actualité de l'intelligence artificielle, décrypté et expliqué chaque jour.