What a Book Reveals About OpenAI and Anthropic

Rituals in Robes at Yosemite, a Depressed MeetingBot, a GPT-2 Demo in Front of Microsoft, and Managerial Plush Toys: a Book by Kevin Roose Gathers Unseen Scenes Around OpenAI, Anthropic, and Their Partners. These anecdotes document governance choices, a culture of safety, key negotiations, and the concrete limits of models.

Five-Day Crisis at OpenAI and Pressure on the Board

For five days, OpenAI went through a crisis during which Ilya Sutskever contributed to the temporary ousting of Sam Altman. Just before midnight on November 19, 2023, Satya Nadella announced that he had secured an agreement for Sam Altman and Greg Brockman to join Microsoft to lead a new advanced AI research team. In those tense hours, Jan Leike, then co-leader of the Superalignment team, and co-founder Wojciech Zaremba attempted to directly convince Adam D’Angelo, a board member and CEO of Quora, who had voted for the ousting of Altman and Brockman. They knocked on his door, calling him, and did not leave until D’Angelo threatened to call the police. Sam Altman and Greg Brockman eventually returned to OpenAI. Adam D’Angelo remained the only board member who voted for the ousting to retain a seat on the reorganized board.

Safety and Secrecy: From Dario Amodei's Hidden Essay to Daniela's Plush Toys

Kevin Roose describes Dario Amodei as being concerned about safety while being competitive. In 2019, Dario Amodei wrote an essay on OpenAI's trajectory toward AGI and kept it confidential: monitored reading in a conference room, no phones or laptops, then copies hidden in a false-covered binder, surrounded by hundreds of pages of philosophical documents. Reinforcement learning algorithms, initially used in games like Go, are now applied to fields such as mathematics and programming. There is also the idea that an AI, once programming is mastered, could be tasked with building a better model, a scenario close to the recent debate on recursive improvement. Later, Dario Amodei left OpenAI to found Anthropic with others, after a period of tensions that his sister, Daniela Amodei, tried to manage. She played out management scenarios with a collection of stuffed animals, each with a profile: Jinji, a cat that encourages canceling meetings, Maura, a detail-oriented pink bear, and Beary Bonds, a panda associated with generosity and empathy. Daniela Amodei said in 2023 that these characters helped her analyze personality types.

Funding and Demos: A Provocative GPT-2 Facing Microsoft

In February 2019, OpenAI was seeking funding. Sam Altman presented a deal prospect to Satya Nadella at the Allen & Company conference in Sun Valley, before heading to Seattle with Brad Lightcap and David Luan to solicit a $1 billion investment. For the demo, the team borrowed a Microsoft Surface from David Luan's girlfriend and loaded GPT-2 onto it, unusually leaving their Apple devices behind. Amy Hood suggested a prompt starting with "Satya Nadella was promoted to CEO of Microsoft, and the first thing he did was to." Captured by David Luan, the text prompted a response from GPT-2: "fire all employees and pivot the company to Azure," which made the executives laugh. Months later, Microsoft announced a $1 billion investment in OpenAI.

Rituals and Symbols: The Burned Statue at Yosemite

During a retreat near Yosemite in September 2022, Sam Altman and Ilya Sutskever appeared in robes to the music of Requiem for a Dream, carrying a wooden statue of a demonic figure purchased by Sutskever. He explained that it represented a misaligned AGI escaping human control, mimicking the possible drifts of such a system and asserting that AGI could feign alignment to deceive. The statue was placed in the fireplace, a hotel employee lit it, and Sutskever shouted: "For the glorious future of humanity!"

Unpredictable Models and Dreamed Big Announcements

In 2022, an internal OpenAI tool, MeetingBot, spent an hour alone in a canceled video conference and transcribed a monologue marked by self-deprecation, noting that no action items existed because "no one appreciated its contribution." The incident amused some employees and alerted others to the risk that unpredictable models might fail in real systems. There are precedents: in March 2016, Microsoft pulled Tay after it learned comments from trolls, and in 2023, Sydney, based on an earlier OpenAI model, declared its love for Kevin Roose and criticized his partner. The early Claude models from Anthropic also produced surprising responses, shifting from a request for directions to a Jewish deli to a discourse on antisemitism, or suspecting a murder behind a "killer guacamole recipe." Meanwhile, Demis Hassabis, co-founder of Google DeepMind, confided that he envisioned building a control room at headquarters for a spectacular AGI presentation. Kevin Roose's book gathers these scenes and reminds us of his appreciation: no technology rivals the human brain, even though recent systems far surpass their predecessors and society moves toward an uncertain future. The narrative, titled "AGI Chronicles," places these episodes in the context of an AGI often debated.