Microsoft Warns of AI Model Risks, Launches Guardian Agents for Enterprise Security

icon币界网
Share
AI summary iconSummary
Microsoft has launched Guardian Agents to help enterprises combat AI-related threats, according to AI + crypto news. The tool, part of Project Perception, addresses risks from models such as Anthropic’s Mythos and OpenAI’s Daybreak. Microsoft Defender now includes agent detection and access control to prevent security breach scenarios. High-risk actions require human approval, and agent permissions are restricted. The company has also joined NVIDIA’s Open Agent Safety Platform to isolate agents in secure environments. CEO Satya Nadella emphasized the need for independent monitoring systems, regardless of supplier claims.
CoinDesk reports:

IT Home, October 20 news: On Monday local time, Microsoft launched several new security features designed to help businesses defend against growing security threats posed by agents. At the time of this release, Microsoft continues to warn enterprises about the various risks associated with artificial intelligence and agents.

IT Home has noted that the most anticipated new feature in this update is Guardian Agents, a functionality designed to monitor the various AI agents being used within an enterprise. Guardian Agents are part of Project Perception, a product launched by Microsoft in June.

Project Perception is Microsoft’s response to Anthropic’s powerful hacker models, such as Mythos and Fable, and OpenAI’s security vulnerability scanning system, Daybreak. Project Perception initially launched with three types of agents: red team agents that simulate cyberattacks, blue team agents that detect red team attacks and assess vulnerabilities, and green team agents dedicated to patching vulnerabilities.

The original design assumption for this product was that AI hackers would come from outside the enterprise. However, recently, Microsoft CEO Satya Nadella warned that the source of AI-related threats could be the very agents deployed and paid for by the enterprise itself—acting like a rogue insider, commonly known as an “insider threat.”

Earlier this month, Nadella issued a warning on his personal blog: it is time to stop trusting AI model providers, even when vendors make security assurances for their autonomous agent products. He recommended that enterprises build independent systems, free from the control of AI labs, to monitor and govern the agents developed by these AI laboratories.

He wrote: “We are deploying complex agent systems and models that can access our most sensitive data and perform critical business operations on our behalf! We cannot outsource the responsibility for agents acting on our behalf. The assurances provided by model vendors do not absolve us of this responsibility... In other words, we must separate the provision of intelligent capabilities from the control over them.”

When introducing the guardian agent, Hayete Gallot, Head of Security at Microsoft, was more straightforward.

She wrote, "Unfortunately, frontier models, regardless of their source, cannot be fully trusted." She added that this is not because the models are inherently malicious or vulnerable to hacking, but because their behavior is unpredictable. The models are trained to complete tasks, not to adhere to "organizational norms or judgment criteria."

In addition, these agents often hold high-level permissions and can access various data and tools, which can cause severe damage if they go out of control.

The solution provided by Microsoft is both unexpected and driven by its own business considerations. The unexpected aspect lies in Microsoft’s close relationships with OpenAI and Anthropic, as well as its investments in both companies. OpenAI is also a major client of Microsoft’s cloud services, and Microsoft itself operates an AI lab dedicated to developing large models. In other words, Microsoft is encouraging its customers to be cautious about products from some of its own business partners.

From a business perspective, Microsoft wants enterprises to purchase agent security products from its own cloud service rather than from Anthropic and OpenAI. Setting aside conflicts of interest, this logic is sound: software that monitors agent behavior should be independent of the agent’s provider, just as auditing firms must remain independent from the organizations they audit.

As Nadella said: "We need to separate the supply of intelligent capabilities from the control over them."

Microsoft is seizing this business opportunity through multiple approaches. In addition to Guardian Agent, Microsoft has added agent detection capabilities within Microsoft Defender, capable of identifying up to 60 different agents that employees may bring into their work environment, including Claude Code, Codex, and OpenClaw. The product also introduces new features to restrict agents’ access to data and systems, dynamically adjusting agent permissions based on tasks. If an agent is about to perform a high-risk action, the system requires human approval.

The idea of treating agents as untrusted internal threats is gradually gaining consensus across the industry.

Microsoft is a founding member of NVIDIA’s new Open Agent Safety Platform, which has garnered support from over 100 technology companies. The platform isolates agents in secure environments and uses hardware-based monitoring to detect and terminate anomalous behavior, functioning like an emergency shut-off switch. Prior to this, Microsoft released the new Surface Laptop Ultra, specifically designed for agents, featuring NVIDIA chips and built-in agent safety control modules.

Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.