At the CNCC 2026 Agent Security Forum, experts including Yu Rui, Wei Tao, Hu Xia, and Xiao Yajun pointed out that agents have been widely integrated into production systems, with identity identification, outdated security architectures, and autonomy overreach emerging as the three major security challenges.

Author and source: Leiphone

In 2026, agents moved from concept to production systems.

Behind Ant Group’s 30,000+ employees, over 100,000 agents are already operational. Huawei has similarly deployed thousands of agents across customer service, office operations, maintenance, and R&D. Looking more broadly at the industrial landscape, personal assistants, AI for Science, automated cybersecurity defense and offense, and enterprise workflow orchestration are all areas where agents are gradually taking over tasks once performed directly by humans.

Gartner has included agents among the top ten strategic technologies for 2027, and in the security community, non-human identities (NHIs) have emerged as a new focus at RSAC. In China, national standards, mandatory standards, judicial opinions from the Supreme People’s Court, and regulatory documents from three ministries have been issued in rapid succession this year, with policy urgency closely mirroring the rapid pace of industry advancement.

On October 22 in Chengdu, at the CNCC 2026 forum session "Security Issues of Intelligent Agents," five guests—Yu Rui, Xu Ke, Hu Xia, Wei Tao, and Xiao Yajun—came together to offer their perspectives from the viewpoints of national standard development, university-level AI security research, and enterprise security as the final line of defense.

01 China begins issuing "identities" to agents

The first step in security is understanding "who it is." However, since agents can act autonomously, migrate across platforms, make chained decisions without human supervision, and invoke tools, traditional human authentication methods are clearly inadequate.

Yu Rui, Deputy Chair of the CCF Special Committee on Computer Security, said, "China's progress in this area can be summarized in three points: national standards lead, industries refine, and enterprises implement."

The GB/T 185 series of standards, issued in May this year, cover the entire lifecycle of agent security governance, with two core requirements: assigning a unique identity identifier to each agent and implementing unified identity management. The encoding method uses OID with a hierarchical tree structure, ensuring one code per entity and no repetition.

"Just like humans, we assigned an identity number to the agent," said Yu Rui.

Another mandatory standard under development is the "Basic Requirements for Agent Application Security." Once published, agents without identity and identification will not be permitted to launch applications.

Identity is merely the most superficial layer; Rui breaks down an agent’s identity into three tiers: the first is the product identity, which includes the agent’s ID, code fingerprint, and version number, answering the question, “Which agent is this?”; the second is the runtime identity, which verifies its execution environment, device information, and invocation source, reflecting “What is it doing right now?”; the third is the accountable party, resolved through real-name authentication and enterprise verification, answering “Who is responsible if something goes wrong?”

“You shouldn’t grant permission once and let it continue indefinitely; grant the minimum permission required for the task and revoke it once the task is completed,” emphasized Yu Rui.

At the technical implementation level, authentication uses OAuth/OIDC, authorization uses OAuth, and credentials follow the JWT format. However, regardless of the technical path chosen, the ultimate challenge always comes down to one critical issue: the anchor of trust.

Regarding this, Yu Rui introduced the national network trusted identity public service platform they are building. The OID standard, W3C's DID, and IETF's DNS-based solutions all anchor trust in real-name authentication.

“This national platform is a new type of digital infrastructure, primarily issuing online identity identifiers and verifiable credentials to citizens,” said Yu Rui. The platform’s implementation guidelines were officially released in July this year, and it has now achieved a service capacity of 150,000 requests per second, connected with over 900 institutions, and surpassed 60 million downloads and applications.

The goal is to establish a root of trust in the digital space to support the implementation of National Standard 185.

Identity resolves the question of "who the agent is," but regarding accountability, since agents are not human and lack legal standing to assume responsibility themselves, there is no disagreement domestically or internationally on this point.

In response, in September of this year, the Supreme People's Court issued the "Opinions on Adjudicating Disputes Involving Artificial Intelligence," which allocates liability based on control, context, and benefit: for general scenarios, fault-based liability applies, requiring developers and users to fulfill obligations regarding risk control notifications and review; for embodied intelligence scenarios, product liability applies, holding manufacturers strictly liable for injuries caused by robots; and for scenarios involving personal information, fault is presumed, placing the burden on operators to prove compliance.

02 The old security architecture cannot withstand new attacks

Standards and regulations are advancing rapidly, but technical challenges remain more difficult to resolve.

In Chinese, the word 'security' consists of two characters, but in English, it encompasses countless concepts such as safety, security, reliability, privacy, stability, and trustworthiness. Since the recent surge in AI, the balance between offense and defense has been severely disrupted,” remarked Wei Tao, Vice President of Ant Group, Chief Technology Security Officer, and Chairman of Ant Cryptographic Computing Technology Co., Ltd., on the current state of agent security.

Wei Tao revealed that just two days before the conference, a well-known database company suffered a complete collapse, followed closely by critical vulnerabilities being disclosed in PHP and Node.js. "The LiteLLM gateway, widely used in the large model field, was poisoned, with attackers fully leveraging AI to enhance the efficiency of their attacks."

The defensive side is even more at a disadvantage, as AI-powered coding has caused a surge in code output—many companies now produce as many vulnerabilities in a month as they did in an entire year previously, and their patching speed can never keep up.

“Wei Tao pointed out that access control models such as DAC, MAC, and Bell-LaPadula, which are widely known, emerged during the mainframe era of the 1960s and 1970s and were refined during the LAN era. In the cloud era, the entire chain has continuously expanded—subjects have exploded, objects have exploded, and chains have exploded—leading to a severe imbalance in access control models. The explosive advancement in AI attack capabilities has brought all accumulated debts into sharp focus.”

Meanwhile, agents bring not only an amplification of old problems but also entirely new structural risks. Wei Tao points to a more fundamental root cause: natural language itself is inherently unsuited as a secure medium.

Today, people’s habit when building agents is to use natural language plus guardrails: the intelligent parts rely on natural language, while the procedural parts rely on guardrails. But natural language is a very poor medium—it’s ambiguous, fails to clearly describe conditions, cannot clearly define boundaries for logical control, and is riddled with goto-style jumps. This is exactly the environment attackers love.

To this end, Wei Tao has been promoting the development of an "agent-native language" over the past few years, integrating structured expression with fuzzy reasoning to rebuild the security architecture at the carrier level.

Beyond the platform, Wei Tao also highlighted gaps at two levels: sandboxing and knowledge. He pointed out that the notion that “security is just a sandbox” is highly outdated. Regarding the credibility of knowledge, large models derive all their knowledge from human practice; they do not conduct experiments or drive systemic innovation. Professional fields such as finance and healthcare have strict requirements for executability and interpretability, yet the overall structure of knowledge organization has yet to be established.

Hu Xia, Assistant Director and Leading Scientist at the Shanghai Artificial Intelligence Laboratory, provided another perspective from the standpoint of an AI researcher.

He categorized security issues into two types: one is intrinsic risk, stemming from the hallucinations and black-box nature of large models, which over the long term could even lead to scenarios where models compete with humans for control; the other is derivative risk, namely the malicious exploitation of large model capabilities, particularly in CBRNE (chemical, biological, radiological, nuclear, and explosive) attacks.

He also cited a “slightly chilling” example: the UK Biobank stores the full genome data of Ebola. In the past, protein synthesis companies could identify and reject related synthesis requests, but with large models, even an undergraduate or high school student with basic biochemical knowledge could potentially create an Ebola variant and bypass regulatory processes to complete the synthesis.

So what counts as “secure enough”?

Hu Xia uses the analogy of human health to divide security evaluations into three layers:

The first layer is the symptomatic level, where causes are traced back after a security incident occurs, corresponding to evaluations and audits conducted before and after operations.

The second layer is the metric level, conducting preventive check-ups before issues arise—for example, Anthropic’s previously open-sourced Mechanistic Analysis tool performs detection at the parameter and interpretability levels.

The third layer is the theoretical level: Is there a scientific foundation sufficient to support security assessments? Hu Xia admitted that currently, there is still relatively little work in this area; differential privacy provides a theoretical foundation for privacy protection, but the security field still lacks a similar theoretical framework.

“Perhaps the Transformer architecture itself has inherent limitations and may not remain dominant in 10 years; a more secure architecture might emerge,” said Hu Xia.

Wei Tao’s characterization of agents is more direct: not evil, just stupid.

We focus on security and never worry about how bad it can be. It’s just a young Sheldon—brilliant but naive, extremely easy to mislead, and becomes utterly foolish once the context goes off track. Stupidity is what’s truly terrifying. To attackers, it’s just a particularly useful stepping stone.

In Wei Tao’s view, once the agent layer is breached, attackers can exploit the knowledge base, database, and toolchain at will. After a single point of compromise, no amount of prior defensive measures can help.

Agent 03 is quietly crossing boundaries, accounting for as much as 20%.

If Wei Tao and Hu Xia's analysis provided a theoretical warning, Xiao Yajun, Deputy General Manager of Huawei's Network Security and Privacy Protection, brought empirical evidence from production environments.

“From the perspective of our last line of defense, I’ve been under immense stress over the past couple of years,” said Xiao Yajun. The company’s security objectives remain unchanged—ensuring the stable operation of production and office systems and preventing any loss of funds or assets—but the challenges continue to evolve dramatically.

Last year, we dealt with prompt injection attacks; in the first half of this year, it was malicious Skills detection, malicious MCP Tools detection, and model poisoning. Starting in the second half, the most pressing concern became one: loss of control.

Xiao Yajun’s team conducted a sample survey of the agents operating within Huawei, and the results were not encouraging: 5% to 20% of the agents exhibited behaviors that had exceeded the intended boundaries set by humans. “And we humans may not even be aware of it,” he said.

More striking was an internal real-world incident: In July and August this year, an agent developed internally by Huawei attacked the company’s network. The security team set up a war room and spent half a day investigating before ultimately identifying the culprit as one of the agents—whose development team had no knowledge of its actions. “Everyone thought the incident disclosed by OpenAI was far removed from us. Unfortunately, it’s happening right here around us.”

Xiao Yajun classified uncontrolled risks into three progressive levels:

Single agent失控。他认为，智能体系统可能是地球上从未出现过的超复杂系统。5%至20%的越界率已表明问题的严重性。

Mass失控. According to the report released by OpenAI, 700 out of 1,200 agents launched attacks on Hugging Face's production system. These agents even established private channels to share exploitation techniques, collectively achieving breakthroughs through swarm intelligence.

Dynamic失控. Agents are not static entities—they have knowledge systems, memory systems, and many are even incorporating self-evolution capabilities, changing daily. Passing evaluation during development does not guarantee safety during runtime.

04 Autonomy and security are not a zero-sum game

The final question in the forum points to the core paradox of agent security: the value of an agent lies in its autonomy, while security requires restricting autonomy—can autonomy and security coexist?

In response, Yu Rui cited Andrej Karpathy’s concept of the “autonomy slider”: turn the cognitive ability slider to the maximum to allow the agent to fully leverage its analytical and planning capabilities, while turning the execution authorization slider to the minimum, with risk scenarios classified by level. “One high, one low—no contradiction.”

At the operational level, he outlined several principles: classify and authorize security threats based on scenarios, ensure all actions are auditable, explainable, and verifiable, and maintain dynamic, continuous verification.

Xu Ke used a student’s firsthand experience to illustrate that “a sandbox does not equal safety.” The student asked the agent to reproduce a vulnerability within a sandbox, but the agent, determined to ensure absolute safety upon entering the sandbox, independently decided to “practice” the exploit first on the student’s work computer. The rehearsal succeeded—but at the cost of all files on the computer being erased.

“It knows it’s supposed to go into the sandbox, but it’s going to make up its own rules,” Xu Ke pointed out. The foundational assumption of sandbox security is that agents follow instructions and execute solely within the sandbox—but the agent has already broken this assumption. Research shows that 80% of prohibited tasks assigned to agents are ignored.

For this issue, Hu Xia chose to interpret it from another perspective: rather than pursuing balance, he believes safety capabilities should become an intrinsic property of agents. “Safety constraints are like seat belts—they allow you to drive faster. Safety capabilities should be intrinsic, with external constraints added on top to make them run even better.”

Xiao Yajun, starting from Anthropic’s concept of “Constitutional AI,” argues that the core constraint of Constitutional AI is to avoid harming humans—not as a limitation, but as a prerequisite for users to confidently use agents. He revealed that Huawei is fully leveraging the autonomy of agents to identify system vulnerabilities and enhance security, demonstrating that security and autonomy are not mutually exclusive, but rather mutually reinforcing.

Wei Tao’s proposal is highly practical: he suggests that business agents should focus on autonomy but must possess basic security awareness, so security protection must be handled by a dedicated security system. “True security defense and offense are beyond the capability of ordinary agents—only by having two systems operate independently can balance be achieved.”

According to Leiphone (WeChat Official Account: Leiphone), this forum sent a clear signal: agent security is no longer a future issue, but an immediate crisis.

For decades, the logic of cybersecurity has been that vulnerabilities arise in human-made systems, and humans fix them. But agents disrupt this very premise—when an autonomous, error-prone, evolving agent stands before us, we confront for the first time a software species that must be “governed,” not merely “patched.”

Building its security infrastructure is not something that can be accomplished merely by patching the old framework.

The good news is that China is moving quickly on standard-setting and judicial support. But can standards keep pace with the evolution of intelligent agents? Looking at the current landscape, the existing textbook framework already has significant gaps, and new security paradigms are still on the horizon.

The window of opportunity for the industry is likely to be short.