Odaily Planet Daily report: Microsoft CEO Satya Nadella published a lengthy article stating that as superintelligence systems evolve, the industry must reassess the trust architecture for AI, cannot simply rely on the promises of model providers, and cannot outsource responsibility for AI-driven actions.
Nadella said that in the past, when traditional software systems were deployed across industries, developers could typically trace behavioral outcomes to specific code paths. However, today’s cutting-edge AI models surpass the capabilities of traditional software systems yet cannot clearly attribute model outputs to specific training data or model weight configurations. Enterprises are integrating AI systems with autonomous action capabilities into sensitive data and granting them authority to perform critical tasks, necessitating the construction of systems with observability, testability, and controllability—separating “intelligence supply” from “permission control.”
Nadella proposed that superintelligence should be governed through an engineering approach, constraining non-deterministic models via deterministic system design, human oversight, and operational protocols, and managing AI systems as potential "internal risks" by limiting permissions, logging behaviors, and establishing isolation boundaries. He also proposed that superintelligence systems should adhere to principles including multi-model collaboration, full observability, continuous validation, independent control, independent auditing, risk isolation, and incident disclosure. Among these, he emphasized that transparency of the model's chain of thought (CoT) is a foundational requirement, but CoT transparency alone is insufficient, as the model's outputs may still lack reliable interpretability.
Nadella concluded that the most trustworthy superintelligent systems in the future will not be the "most trustworthy models," but rather "systems that can operate safely even without fully trusting the model."