Nadella Calls for Human Audit and Halt of AI Systems

The Microsoft CEO defends an AI governance model focused on safety: diversity of models, complete traceability, independent auditing, and the ability to shut down systems. He also claims that current models remain opaque while accessing sensitive data, and adopts the term "Super Intelligence." Google DeepMind had already promoted a similar internal security approach.

Concrete Safeguards: Diversity, Logging, Auditing, and Shutdown

Satya Nadella proposes to dissociate intelligence from the control exercised over it, in order to limit the trust required in the model. He recommends prioritizing diversity of models rather than dependence on a single one. He advocates for comprehensive logging of actions taken by systems and independent auditing. He emphasizes the need to allow humans to shut down systems at any time. According to him, the most reliable system is not the one with the best model, but the one that requires the least trust in that model. He believes that the architecture should incorporate containment and control mechanisms.

What He Criticizes About Current Systems: Opacity and Critical Access

Satya Nadella asserts that current AI systems do not allow for understanding why they produce a given response or decision. He points out that they are nonetheless deployed with access to sensitive data and are authorized to take critical actions. He writes that a sufficiently capable actor with access to important systems can make mistakes or be compromised, which justifies the need for a containment and control architecture. He clarifies that the models are not necessarily malicious, but recommends treating them as internal threats. He presents his proposals from a cybersecurity perspective.

Terminology and Context: "Super Intelligence" and Precedents

Satya Nadella uses the term "Super Intelligence" in his latest writing, echoing the terminology favored by Donald Trump, who stated that those who continue to use "Artificial Intelligence" should be considered enemies. The idea of treating AI models as internal threats had already been put forward by Google DeepMind. In the past, Nadella has argued that AI models are becoming interchangeable commodities and has accused AI companies of hypocrisy during the crackdown on distillation. These positions align with the trajectory of the Microsoft CEO.