Meta and OpenAI: Inside the AI Labs

A Career Between Meta and OpenAI

Prakhar Agarwal, a researcher at Meta's Superintelligence Labs, shares his unique journey that led him to work in two of the most influential laboratories in the field of artificial intelligence: Meta and OpenAI. Formerly at OpenAI, his experience provides a rare insight into the dynamics and challenges faced in these cutting-edge environments.

The Daily Challenges of Laboratories

Each workday is different, influenced by the stage of the ongoing project and the deliverables to be produced. At OpenAI and Meta, projects are marked by key milestones, such as training sessions or reinforcement learning phases, which can span a period of ten months. As deadlines approach, the pressure increases significantly.

The work relies on the current iteration of the model. If a model has shortcomings in a particular area and a solution is found, it must be applied before the next version. Missing this deadline means that the same issues might not be addressed in the next version, rendering the solution obsolete.

When deadlines are still far off, the focus is on evaluating the model's performance and identifying failure cases. The work is highly dynamic: some tasks seem simple but turn out to be complex, sometimes requiring a week of effort due to numerous unknowns.

Resources and Collaboration in Laboratories

In these advanced research laboratories, the main constraint is computing power. Unlike large tech companies where staffing can be easily scaled, here, each researcher needs computing resources to make progress. An increase in the number of researchers means a division of these resources, thereby slowing down each individual's progress.

It is crucial to have high-bandwidth communication among stakeholders. It must be direct and unmediated to maintain a rapid iteration speed. Teams are often small and close-knit, fostering smooth collaboration. Each member has their own projects but frequently collaborates with others on joint initiatives. Sometimes, Prakhar collaborates more with people outside his immediate team than with those within it. At Meta and OpenAI, teams are primarily composed of experienced researchers, which broadens the scope of individual projects.

The Importance of Communication and Code

Communication is central in these laboratories. Much information is undocumented, making it essential to clearly explain one's actions, the underlying reasons, and the next steps. Sharing results and receiving feedback is indispensable for progress.

Mastery of code is also crucial. Code often evolves faster than documentation, requiring the ability to understand and adapt quickly. A good understanding of different domains allows for a comprehensive view of ongoing ideas and approaches, and helps find effective ways to contribute.

Learning from Failures

A research paper highlights what works, but it does not reveal the many unsuccessful attempts that preceded it. Before doing X, Y, and Z, Prakhar tried 50 different things that did not work. In these laboratories, experimentation is constant, and failures are as instructive as successes. Teams thus develop a strong intuition about what is likely to work or not.

Outside observers often focus on successes but underestimate the value of failures, which are essential for progress.

Advice for Aspiring Researchers

Managing burnout is complex and cannot be solved with a simple formula. Working at the cutting edge of technology involves keeping pace with the rapid innovations. To succeed, it is crucial to be open to exploring new ideas and not to limit oneself to current skills.

It is important to develop the ability to adapt to entirely new subjects. Sometimes, the challenge is more psychological than technical, and one must be ready to embrace the unknown to advance in these constantly evolving environments.