OpenAI Launches AI Futures and Its Governance Principles

OpenAI launches AI Futures, a showcase for its new Strategic Futures team, to address the risk of power concentration in the era of advanced AI systems. The team puts forward a series of principles, including the traceability of high-stakes actions to a human responsible, combined with a defense of anonymity and privacy. It announces interdisciplinary work and long-term publications.

Traceability of Risky Actions and Anonymity Established as Principles

At the heart of OpenAI Strategic Futures' governance is the principle of "limited readability": in cases where AI systems perform high-impact actions concerning the physical integrity or property of others, it must be possible to link these actions to a physical person or an organization under human control. Simultaneously, the team emphasizes a design of these systems that prioritizes privacy protection and asserts that freedom of expression entails anonymity, including the ability to use AI without revealing one's identity in many contexts. It also frames collective action as necessary but limited to a small category of risks, of modest and limited scope, and calls for the law to prioritize the empowerment of individuals and small organizations over centralization. Finally, it asserts the primacy of human political, social, and economic institutions to guide global affairs, even as these institutions must evolve.

A Doctrine of Power Balance Rather Than Radical Decentralization

The team believes that an appropriate response to the structural challenge posed by advanced machine intelligence is probably not a maximal decentralization of power. It advocates for a balance that prevents a single actor, or a small group, from dominating others, in the spirit of a system of checks and channelled ambitions. The reasoning presented connects with contemporary notions of incentives and game theory, while also drawing from an older tradition: the American Founding Fathers did not intend to abolish all concentration but to organize forces in tension, illustrated at the time by metaphors borrowed from Newtonian mechanics and a vocabulary of orbits and spheres. James Madison is cited to remind that purely written protections are insufficient and that the fight against undue concentration was central.

The Priority Risk: Power Concentration

OpenAI Strategic Futures asserts that, in the long term, power concentration represents the most significant, serious, and conceptually thorny category of risks among the safety and policy issues of AI. Avoiding a shift that undermines individual autonomy is posed as an imperative, with no scientific, technological, or economic progress justifying such a lasting sacrifice. The team links this power knot to concerns already visible among companies, interest groups, and individuals, who fear being sidelined from decision-making spaces. It presents this issue as structural, transcending partisan divides and institutional rituals, and argues that individuals could lose agency even if electoral procedures continue.

Envisioned Scenarios: Force Without Troops and Revenue Streams from Data Centers

The team considers it plausible that an era could emerge where the projection of force, both domestically and internationally, would no longer depend on cooperative soldiers or police due to advanced autonomous systems. It suggests that public revenue collection could, with advancements in machine intelligence, shift from human labor to data centers, while entire segments of bureaucracy would be automated. In these scenarios, the perpetuation of power would no longer rely on broad social consent, and the public's role in negotiation could significantly diminish or even disappear.

Work Program: Iterative Publications and Interdisciplinary Research

OpenAI Strategic Futures indicates that it will disseminate its analyses on AI Futures as well as through other media, such as articles, videos, and podcasts. The initiative is described as progressive, welcoming discussion and feedback, explicitly relying on forecasting and conducting work at the intersection of public policy, economics, law, history, and machine learning. According to the team, it faces more questions than certainties regarding potential developments, believes that the mere existence of written guarantees is insufficient, while also acknowledging the importance of constitutional principles. It claims ideals of free expression and individual liberty and asserts that no isolated entity can, on its own, resolve the major questions raised, calling for a collective approach from humanity.

Organizational Announcement: AI Futures Becomes the Showcase for Strategic Futures

OpenAI officially designates AI Futures as a dedicated blog for its new Strategic Futures team. This small team places at the center of its mission a guiding question: how to adapt the structures of a free society to preserve the rights and autonomy of individuals in the face of potentially transformative AI.