Token Factory is a next-generation AI infrastructure with Token throughput as its core output metric, marking the shift of the intelligent computing industry from resource competition to efficiency competition. Under the scaling law, the former north star metric for intelligent computing centers was GPU resource accumulation; however, the market ultimately demands productive Tokens, making scheduling capability and inference optimization equally critical. The resource conversion efficiency of Token Factory is determined by scheduling platforms and chip-model collaboration within the AI system software stack—advanced scheduling systems such as NVIDIA DCGM, CoreWeave NeoCloud, and Alibaba Cloud Aegaeon have already demonstrated their value. Chip-model collaboration enhances Token output efficiency per unit of computing power through two pathways: co-designing proprietary chips with models or integrating with chip vendor ecosystems. As GPU hardware gradually standardizes, competitive advantages in AI infrastructure will increasingly stem from software system capabilities.Author and Source: Changjiang Securities Research Report
Key Report Points Token Factory is a next-generation AI infrastructure with Token throughput as its core output metric. Its operational goal is no longer simply to provide GPU computing power, but to efficiently convert electricity, GPUs, networks, and models into a continuous stream of Tokens, ultimately transforming them into intelligent services and revenue. Since Token throughput directly determines an AI factory’s revenue potential and return on investment (ROI), the focus is shifting toward maximizing the number of Tokens produced per unit of GPU and per unit of power consumption under limited resources. In this process, the competitive logic is evolving from “how many GPUs you own” to “how effectively you can make GPUs produce more Tokens.”
The intelligent computing industry is transitioning from resource competition to efficiency competition.
Under the backdrop of scaling laws, the north star metric for AI computing centers in the past was maximizing GPU deployment—those with the most GPUs were seen as holding the industry’s core competitive advantage. However, the market has gradually realized that possessing a large number of GPUs may only represent an initial entry barrier, not sufficient to build long-term competitive value. This is because GPUs are merely one of the initial means of production; what the market ultimately needs are productive Tokens. Producing Tokens requires many other capabilities—such as electricity, network infrastructure, scheduling efficiency, and inference optimization. For AI computing centers, merely accumulating GPUs while lacking in other areas can lead to significant resource waste at the same scale. Against this backdrop, Token factories have emerged as a natural evolution. A Token factory refers to the transformation of data centers in the AI inference era from traditional “data storage warehouses” into industrialized facilities dedicated to producing the fundamental units of AI-generated content—Tokens.
Token Factory: A next-generation AI infrastructure with token throughput as the core performance metric
Token Factory is a next-generation AI infrastructure with Token throughput as its core performance metric. Its operational goal is no longer simply to provide GPU computing power, but to efficiently convert electricity, GPUs, networks, and models into a continuous stream of Tokens, ultimately transforming them into intelligent services and revenue. As NVIDIA’s Jensen Huang has stated, the essence of an AI factory is to convert energy into Tokens, and Tokens into tangible value—with Tokens increasingly becoming the fundamental unit for measuring AI productivity. Since Token throughput directly determines an AI factory’s revenue potential and return on investment (ROI), the ability to maximize Token output per unit of GPU and per unit of power consumption has become the new frontier in AI infrastructure competition. In this process, the competitive logic is shifting from “how many GPUs you own” to “how efficiently you can make GPUs produce more Tokens.”
The AI system software stack determines the resource conversion efficiency of the Token Factory.
In the AI era, GPUs have gradually become standardized computing resources, with their performance determining the theoretical upper limit of a Token factory’s production capacity. What truly determines how much value can be extracted from a given amount of computing power, however, is the AI software stack encompassing scheduling platforms, inference engines, compilers, and model optimization. Unlike the traditional cloud era, which primarily relied on hardware scaling to boost computing power, Token factories emphasize continuously improving the production efficiency of existing computing resources through software—producing more Tokens with fewer GPUs and lower power consumption. This report will primarily discuss two core capabilities: 1) Centered on the scheduling platform, leveraging various technologies to increase GPU utilization, reduce resource fragmentation and idle time, and ensure more GPUs are actively engaged in Token production; 2) Centered on co-design between chips and models, jointly optimizing chip architecture, compilers, inference frameworks, and model structures to enhance Token generation efficiency per GPU and performance per unit of power, further unlocking hardware potential.
From industry practices, the benefits brought by advanced scheduling systems and chip-module collaboration have become evident.
From industry practice, the benefits brought by advanced scheduling systems and chip-model collaboration have been increasingly validated in production environments. In recent years, whether cloud providers such as CoreWeave, Google, and Alibaba Cloud, or model providers like Deepseek, all have shifted their optimization focus from merely improving hardware performance to enhancing overall system efficiency. Looking ahead, as GPU hardware gradually standardizes, competitive advantages in AI infrastructure will increasingly stem from system software capabilities. Overseas players such as NeoCloud, cloud providers, model developers, and chip manufacturers will all continue to build differentiated competitive advantages around scheduling platforms, inference engines, and chip-model collaboration.
The emergence of token factories signifies that the intelligent computing industry is transitioning from a phase of resource competition to one of efficiency competition—from "accumulating resources" to "maximizing resources"—potentially leading to a fundamental shift in the industry's north star metric.
Under the backdrop of scaling laws, the North Star metric for AI computing centers in the past was maximizing GPU deployment—those with the most GPUs were seen as holding the industry’s core competitive advantage. However, the market has gradually realized that having a large number of GPUs may only represent an initial entry barrier, not sufficient to build long-term competitive value. This is because GPUs are merely one of the initial production inputs; what the market ultimately needs are productive tokens. Producing these tokens requires many other capabilities—such as electricity, network infrastructure, efficient scheduling, and inference optimization. For AI computing centers, merely possessing a large number of GPUs while lacking in other areas can lead to significant resource waste at the same scale of investment.
Against this backdrop, the Token Factory has emerged accordingly. A Token Factory refers to a data center that, in the era of AI inference, transforms from a traditional “data storage warehouse” into an industrial facility dedicated to producing the fundamental units of AI-generated content—Tokens. This concept was first introduced by NVIDIA CEO Jensen Huang in 2024 and systematically elaborated upon at the 2026 GTC conference through the framework of “Token Factory Economics.” In terms of industrial positioning, a Token Factory occupies a role intermediate between pure compute providers and downstream enterprises, occupying a similar position in the value chain as cloud computing.
Token throughput directly determines the revenue of the Token factory.
The emergence of token factories may signify a formal shift from the past industry logic centered on "accumulating resources" toward one focused on "extracting resources," and the north star metrics (ultimate goals) corresponding to different business models have also undergone fundamental changes.
The revenue formula for the Token Factory is: Token usage volume × Token pricing, with both factors jointly determining the factory's total revenue. From this formula, we see that while data centers previously focused primarily on the quantity of resources, the north star metric for Token Factories has now shifted to Token production efficiency under limited resources (commonly measured in tokens/s or tokens/task). Those who can generate more Tokens with the same level of resource ownership will earn higher revenue.
How can the efficiency of "resource extraction" be improved? Compared to simple hash power providers, the core incremental capability of a Token factory lies in its proprietary hash power scheduling platform. Both industry case studies and academic experiments have repeatedly validated the importance of scheduling platforms in enhancing the efficiency of Token factories.
The AI system software stack optimizes resource conversion efficiency scheduling for the Token Factory: the importance of AI cloud scheduling exceeds that of traditional cloud.
If scheduling in the traditional cloud era was merely “resource scheduling,” then scheduling systems in the AI cloud era are “multi-dimensional bottleneck systems.” Requests in the traditional cloud era typically exhibit short, independent, and relatively predictable characteristics (such as a surge in traffic during Double Eleven, but still within a predictable range); the CPU cloud scheduling system’s primary task is simply to assign tasks to machines. Unlike traditional cloud systems, which primarily optimize CPU utilization, AI cloud systems face unique challenges such as GPU fragmentation, KV cache fragmentation, and gang scheduling. As a result, the importance of scheduling systems has become even more critical in the AI era.
From real-world industry usage, issues such as GPU resource fragmentation are common. According to industry statistics released by TechTarget in March 2026, when participants were asked about the most critical challenges they face in managing AI task costs, 23% selected "suboptimal CPU and GPU utilization," and 21% selected "fragmented and complex cost structures," both of which reflect actual challenges encountered in AI cloud usage.
Major players in domestic and international GPU cloud and intelligent computing centers have made diverse attempts at various levels.
How can the above issue be resolved? Major domestic and international GPU cloud and intelligent computing center players have made diverse efforts at various levels, exploring solutions ranging from resource monitoring, resource sharing, cluster scheduling, to token-level scheduling.
NVIDIA: DCGM System, Solving the Black Box Issue in GPU Scheduling
At the resource observation layer, NVIDIA introduced DCGM (Data Center GPU Manager), which collects key metrics in real time—such as GPU utilization, HBM memory usage, NVLink bandwidth, power consumption, and temperature—to provide a unified data foundation for scheduling systems. Although DCGM itself does not perform scheduling, it solves the problem of GPU clusters being “invisible,” offering critical insights for subsequent resource optimization. In other words, DCGM’s role in AI cloud scheduling is analogous to that of sensors in an autonomous driving system.
CoreWeave: Cluster efficiency is key to NeoCloud
At the cluster scheduling layer, NeoCloud providers like CoreWeave are making cluster efficiency a core competitive advantage. CoreWeave’s public data shows that, with a theoretical value of 100%, current industry large model training clusters typically achieve only 35%-45% Model FLOPs Utilization (MFU), indicating significant efficiency waste.
CoreWeave believes that AI infrastructure efficiency losses stem from systemic issues such as communication overhead, cluster management, node failures, data transfer, and scheduling—not merely from the GPUs themselves. Through system-level optimizations like topology-aware scheduling, automated node management, and deep observability, CoreWeave significantly reduces communication and scheduling overhead in GPU clusters, thereby substantially improving MFU for large-scale training workloads. In specific experimental environments, CoreWeave’s platform demonstrated a notable improvement in MFU compared to the industry average.
Alibaba Cloud: Launched the pooling system Aegaeon, which reduces resource waste by advancing scheduling granularity from the GPU level to the token level through token-level auto-scaling technology.
By the end of 2025, a collaboration between Alibaba and Peking University introduced the latest GPU pooling system, Aegaeon, which significantly reduces GPU usage for specific tasks through token-level automatic scaling, thereby greatly improving resource utilization.
Before Aegaeon, mainstream LLM serving systems (such as vLLM, Triton, and ServerlessLLM) had several key characteristics: 1) GPUs were allocated on a model-centric basis, leading to underutilization of resources for large models and overallocation for small ones, resulting in GPU waste; 2) scheduling occurred at the request level rather than the token level, causing overly coarse granularity and frequent queuing. Aegaeon transforms LLM inference from a request-based GPU allocation model to a token-level scheduling system that dynamically shares GPUs across multiple models, significantly improving GPU utilization and reducing the required number of GPUs.
The core innovation of the Aegaeon system lies in its use of token-level automatic scaling, rather than scheduling at the request level as in existing systems. Specifically, the system dynamically decides whether to switch models at the time each token is generated, rather than waiting until the entire request is completed. This innovation enables GPUs to efficiently handle requests for multiple models, significantly reducing wait times between model switches.
Results show that, based on the paper’s data, Aegaeon can support 2 to 2.5 times the request arrival rate and achieves 1.5 to 9 times higher goodput compared to existing systems such as ServerlessLLM and MuxServe.
Core-Model Co-Optimization: Enhancing Token Production Efficiency Core-model co-optimization is increasingly becoming a key direction for improving the production efficiency of AI-powered token factories. As large model training and inference enter the stage of large-scale deployment, merely upgrading GPU hardware can no longer sustainably enhance overall system efficiency. Co-optimization between model architecture, compilers, inference engines, and chips is emerging as a new performance breakthrough.
The industry is converging on two primary development paths: one involves large companies developing their own AI chips and co-designing them with their proprietary models to achieve end-to-end optimization across chips, software stacks, and models; the other entails deep ecosystem partnerships between large model providers and chip manufacturers, tailoring their solutions to specific hardware platforms and jointly optimizing across operators, compilers, inference frameworks, and model architectures to fully unlock hardware performance.
Regardless of the chosen model, the core objective is to enhance token output efficiency per unit of computing power and performance per unit of power consumption through co-design of chips and models, thereby further improving the overall production efficiency and return on investment (ROI) of the token factory.
At the 2026 Alibaba Cloud Summit, Alibaba Cloud achieves comprehensive upgrades in "chip-cloud-model-inference."
At the 2026 Alibaba Cloud Summit, Alibaba unveiled the Panjiu AL128 super-node server powered by the new-generation AI chip, Zhenwu M890, featuring the company’s proprietary interconnect chip, ICN Switch 1.0. This configuration enables 128 AI chips to function as a single computer with P2P latency under 150ns, designed to address massive concurrent inference and large model training demands in Agent scenarios. The debut Zhenwu M890 chip employs a proprietary parallel computing architecture, integrates 144GB of memory, and delivers three times the performance of the Zhenwu 810E, with inter-chip interconnect bandwidth reaching 800GB/s. Native support for multiple data precisions—from FP32 to FP4—enables full-scenario applications including high-precision training and low- or ultra-low-precision inference. In low-precision inference scenarios, the chip ensures model output quality while significantly reducing computational costs per inference.
The Bailian Inference Platform delivers ultimate elasticity. On the Bailian Inference Platform, Alibaba has built a large-scale GPU resource cluster and achieves efficient resource utilization through pooled scheduling. Building on this, Bailian eliminates redundant computations via context caching and employs throughput-aware elastic scheduling to handle traffic spikes, troughs, and load fluctuations. For performance optimization, Bailian integrates Agentic RL—a reinforcement learning mechanism driven by agent-based feedback—to continuously refine model performance. Additionally, Bailian incorporates built-in security governance capabilities to ensure that autonomous agents always operate within defined boundaries.
DeepSeek's deep integration with domestic chips
Accelerated Compatibility with Domestic Chips: On April 24, DeepSeek-V4 released its preview version, achieving leading performance domestically and in the open-source community in terms of 1M context length, agent capabilities, world knowledge, and reasoning. On the day of release, eight domestic chip manufacturers—Huawei Ascend, Cambricon, Hygon, Moore Threads, Musen, Kunlun Tech, Pingtouge Zhenwu, and TianShu Intelligence—announced Day-0 simultaneous compatibility.
