Scott Wiener Aims to Regulate AI in Congress

Elected in San Francisco to Nancy Pelosi's seat, Scott Wiener arrives in Washington with a clear focus on AI: a strong federal regulator, audits, protections for whistleblowers, and a kill switch. He rejects broad federal preemption of state laws and warns about recursive self-improvement. However, he describes himself as only cautiously optimistic, caught between a mobilized opinion against data centers and the uncertainties surrounding Donald Trump.

Local Opposition to Data Centers and Federal Uncertainties Under Trump

Scott Wiener observes a gap between his vision of the risks associated with advanced AI and the national concern, which mainly focuses on opposition to data centers. In Tuesday's vote, voters approved 40 out of 41 ballot measures against data centers, a result that might seem favorable to proponents of AI regulation. However, Wiener believes this movement is primarily driven by local concerns, such as neighborhood nuisances, rising electricity bills, or water usage, which he considers the main reasons for opposition based on his observations.

He expresses only cautious optimism regarding Congress's ability to regulate AI, due to the difficult history of tech regulation. He voices doubts about the possibility of passing legislation under Donald Trump, who labeled the catastrophic risk of AI a "hoax" in September and has shown a willingness not to cooperate with China, while Wiener believes such cooperation is essential to avoid the worst consequences. Nevertheless, he sees the president's unpredictability as a fragile source of hope. Wiener notes that no major catastrophe has occurred so far and hopes that one will not be necessary to provoke a political reaction; if it were to happen, he wishes for the impact to remain limited.

In Congress, a Pro-Risk Agenda with a Dedicated Regulator and Oversight

If a special committee on AI is created in Congress — a possibility discussed by Democrats since September — Scott Wiener says he is ready to participate and supports several proposals under discussion. He advocates for the establishment of a strong, empowered, and agile federal regulator, citing the proposal from Representative Ro Khanna. He emphasizes the need for protections for whistleblowers, testing and audit requirements, a kill switch for AI systems, and increased use of Congress's oversight powers to hold AI labs accountable, ensuring the public is informed and able to voice their opinions.

Wiener opposes broad federal preemption of state laws on AI, recalling Brandeis's idea that states serve as laboratories and must retain their agility, sometimes surpassing that of Congress. He calls for federal policies that have concrete effects and are commensurate with the urgency, taking seriously the possibility of an AI-related catastrophe. According to him, the risk of losing control is high, and the issue of recursive self-improvement must be addressed, or there is a danger of total loss of control. He had hoped for decisive action from Congress before the end of the year but believes that is not materializing and calls for a slowdown in what he considers very rapid development.

In California, Two Foundational Bills, One Vetoed and One Signed

Scott Wiener's action on AI regulation began in California. In 2024, he drafted Bill SB 1047, presented before its veto as the first major legislation on advanced AI in the United States, but it was blocked by Governor Gavin Newsom. He then sponsored SB 53, a less stringent measure, which the governor signed. These two laws have inspired proposals at the federal level.

Wiener explains that he has increasingly turned to AI in recent years, incorporating many concerns from the AI safety movement. In a 20-minute interview, he mentioned recursive self-improvement several times — about six times — which lies at the heart of his concerns.

Pro-Regulation Funding and Criticism of Industry Ties

His campaign benefited from over $3 million contributed by Public First Action, a nonprofit organization supporting AI regulation and largely funded by Anthropic. This financial support has led to accusations from his opponents, who deem him too closely tied to the AI industry.

Scott Wiener responds that Public First supported him because he was, in his view, the first legislator in the country to introduce AI regulation and to have it adopted and enforced. He describes these attacks as cynical and false regarding his proximity to Big Tech.

Warning on Recursive Self-Improvement After METR Report

The day before his interview, the organization METR released a report widely interpreted as indicating that Anthropic may have achieved recursive self-improvement — a process where an AI contributes to its own advancement, with a risk of losing control. Scott Wiener deemed the situation very concerning and cited this report to underscore the urgency for regulation.

As a state senator in California and winner of a tight election for Nancy Pelosi's seat in San Francisco, he believes he can play a key role in federal AI policy. He expressed hope for decisive action from Congress before the end of the year and spoke out as soon as Friday after his victory, just hours after the results were announced.