This article will deeply analyze the current landscape of self-developed AI chips by global tech giants, arguing from economic and underlying technical perspectives why heavy investment in proprietary chip development is an inevitable choice for technology giants, and project the challenges they face and the ultimate future of computing power格局.

Article author and source: 0x9999in1, ME News

Introduction

Since the explosion of generative AI at the end of 2022, the global tech industry has entered a new arms race centered around large models. In this race, computing power has become the foundational infrastructure determining success or failure. NVIDIA, leveraging its long-standing expertise in general-purpose graphics processing units (GPUs) and its absolute dominance over the CUDA software ecosystem, has captured the most substantial profits from this AI wave. However, gross margins exceeding 70% and chronic supply shortages have plunged global cloud service providers (CSPs) and AI giants into a state of computing power anxiety—feeling as though they are working for NVIDIA.

Under this context, ME News Think Tank observes that global tech giants are accelerating their strategic shift toward underlying hardware, sparking an unprecedented wave of "in-house AI chip development." From North American giants like Google, Amazon, Microsoft, and Meta, to Chinese leaders such as Huawei, Baidu, Alibaba, and ByteDance, big tech's chipmaking has evolved from early experimentation into a core strategic imperative critical to their survival. This article provides an in-depth analysis of the current landscape of in-house AI chip development by global tech giants, examining from economic and foundational technical perspectives why heavy investment in proprietary chips is an inevitable choice for these giants, and projecting the challenges they face and the ultimate future of computing power architecture.

Current status of global leading tech giants developing their own AI chips

Currently, major global companies' proprietary AI chips are primarily focused on the cloud (data centers), divided into two main scenarios: training and inference. Unlike NVIDIA’s approach of pursuing generality, these companies predominantly use application-specific integrated circuit (ASIC) architectures, trading off some generality to achieve optimal energy efficiency for specific internal business use cases.

The four major North American cloud providers (the four hyperscalers) entered the self-developed chip market at different times, but have now fully joined, creating a "hedge pool" against reliance on NVIDIA's computing power.

Google: The absolute pioneer and benchmark of in-house development

Google was among the first global giants to recognize that deep learning demanded entirely new hardware requirements. As early as 2015, Google deployed its first-generation Tensor Processing Unit (TPU) internally. After nearly a decade of iteration, the current TPU has evolved into its sixth generation (Trillium).

Google's advantage lies in its closed-loop ecosystem: from the underlying TPU hardware, through the intermediate XLA compiler and JAX framework, to the top-tier Gemini large models. This end-to-end software-hardware integration enables Google to train world-leading multimodal large models without relying on NVIDIA GPU clusters. The significant improvements in interconnect bandwidth and high-bandwidth memory (HBM) with TPU v5p and Trillium demonstrate that Google has now achieved cluster networking capabilities comparable to NVIDIA's NVLink.

Amazon (AWS): Centered on cost and customer choice

AWS's journey into chip design began in 2015 with the acquisition of Annapurna Labs. In the AI space, AWS has developed two product lines: Trainium, focused on training, and Inferentia, focused on inference. AWS’s strategy is highly pragmatic: it does not aim to fully replace GPUs with its own chips, but rather to provide cloud customers with cost-effective computing options. According to AWS official data, running large model inference on the Inferentia2 chip delivers 50% higher performance per watt compared to equivalent Amazon EC2 instances.

Microsoft and Meta: From Passive Payers to Active Breakthroughs

Microsoft and Meta were previously the largest buyers of NVIDIA's H100 and A100 chips. To support OpenAI's training needs and its own Copilot business, Microsoft officially launched its custom AI accelerator chip, Azure Maia 100, at the end of 2023. The chip is built on TSMC's 5nm process and is specifically designed for cloud-based training and inference.

Meta’s path closely aligns with its own business. Its MTIA (Meta Training and Inference Accelerator) chip was initially designed specifically for deep learning recommendation models (DLRM) to optimize ad delivery on Facebook and Instagram. With the surge of the open-source Llama series of large models, the next-generation MTIA chip significantly enhances support for generative AI inference, aiming to reduce the exorbitant inference costs associated with billions of calls.

Unlike North American giants seeking to reduce costs and improve efficiency, Chinese tech giants are developing their own AI chips with a strategic focus on "bottom-line defense" and "supply chain security" in response to U.S. export controls on high-end AI chips.

Huawei: The Pillar of Domestic Computing Power

The Huawei Ascend series is currently the only domestic product capable of replacing NVIDIA A100/H20 in ultra-large-scale cluster training. The Ascend 910B employs the Da Vinci architecture and is deeply integrated with domestic frameworks such as MindSpore through the CANN (Compute Architecture for Neural Networks) layer. Currently, more than half of China’s leading large models, including iFlytek’s Spark and Zhipu AI, have completed or are in the process of adapting and training on the Ascend computing foundation.

The pragmatic approach of internet giants: Baidu, Alibaba, and ByteDance

Baidu's Kunlun芯 is among China's earliest self-developed AI chips deployed by major internet companies and has now evolved to its third generation, fully supporting inference and partial fine-tuning for the Wenxin Yiyan large model. Alibaba's Pingtouge Semiconductor has launched the HanGuang 800, primarily focused on high-concurrency inference scenarios within Alibaba such as e-commerce search and image recognition. ByteDance, as the global company with the largest demand for recommendation algorithm computing power, although entering the field later, is actively collaborating with industry giants like TSMC and Broadcom to customize dedicated AI ASIC chips to meet the massive daily inference demands of its Doubao large model and TikTok/Douyin platforms.

Table 1: Overview of AI Chip Strategies by Major Global Tech Giants

In-depth analysis: Why must major corporations invest heavily in developing their own AI chips?

Developing an AI chip using advanced processes such as 5nm or 3nm involves fabrication costs running into tens of millions of dollars, along with the need to sustain large teams dedicated to chip design and software verification. Despite these formidable barriers, tech giants continue to rush in—ME News Think Tank believes this is driven by a clear business logic, supply chain dynamics, and fundamental technological principles.

The current business model for generative AI is facing a severe inversion risk. Training a large model at the GPT-4 level with trillions of parameters requires tens of thousands of H100 GPUs running for months, resulting in hardware capital expenditures (CapEx) of hundreds of millions of dollars alone. After deployment, ongoing inference costs become an endless sink.

NVIDIA's valuation of over a trillion dollars essentially amounts to imposing a high "compute tax" on the entire AI industry. General-purpose GPUs must accommodate functions like graphics rendering (Graphics) and double-precision floating-point operations (FP64), which consume significant transistor area on the chip but serve no purpose in pure deep learning (which primarily relies on FP16, FP8, or even INT8). When large companies purchase GPUs, they are effectively paying for these unused "dark silicon" regions.

By developing their own custom ASIC chips, large companies can eliminate all redundant functions, dedicating every square millimeter of silicon to tensor computation and memory bandwidth optimization. According to industry estimates, in specific large-scale inference scenarios, the per-computation cost (TCO) of custom ASICs is only one-third to one-fifth that of general-purpose GPUs. For companies like Meta and ByteDance, which require hundreds of billions of inference calls daily, the annual operational expenditure (OpEx) savings once these custom chips are deployed at scale could amount to billions of dollars. Investing hundreds of millions of dollars in chip R&D to achieve tens of billions in cost savings represents a highly certain economic proposition.

Beyond cost, the fragility of the supply chain is the sword of Damocles hanging over tech giants. NVIDIA holds absolute control over production allocation, and delivery times for core GPUs like the H100 and B200 often stretch for months. Without computing power, the AI initiatives of major companies come to a halt.

Developing proprietary chips fundamentally enhances bargaining power against NVIDIA. Even if major companies cannot fully摆脱 NVIDIA in cutting-edge model training, using self-developed chips for inference and recommendation systems significantly reduces absolute dependence on external general-purpose GPUs, thereby gaining more leverage in procurement negotiations.

For Chinese enterprises, developing chips in-house is an unavoidable necessity. Restricted by the U.S. Department of Commerce’s BIS export control regulations, domestic companies cannot access commercial chips with the highest compute density and interconnect bandwidth. Building an autonomous and controllable computing infrastructure through in-house development combined with domestic or non-U.S. semiconductor foundry systems is the only path to ensuring national AI sovereignty and the continuity of enterprise data center operations.

Today, as Moore's Law gradually slows, the marginal gains in performance from purely advancing chip manufacturing processes—such as moving from 5nm to 3nm to 2nm—are diminishing. Future breakthroughs in computing power will increasingly rely on "software-defined hardware" and "hardware-software co-design."

General-purpose GPUs must be compatible with thousands of different applications and algorithms, so their architecture must be generic. In contrast, proprietary chips developed by major companies can be specifically optimized based on the data types, sparsity characteristics, and communication patterns of their core large models.

For example, if a major company’s business focus is on long-context large language model inference, its custom-designed chips can be optimized to maximize on-chip SRAM capacity or HBM memory bandwidth, rather than blindly pursuing peak FLOPS. This ability to “burn in” its own AI algorithms into the underlying silicon enables the creation of performance experiences that competitors cannot easily replicate, thereby building a deep moat.

Real challenges and breakthrough strategies for self-developed AI chips

Although the strategic value of developing in-house AI chips is undeniable, it is by no means a smooth path. Blindly manufacturing chips could not only lead to massive financial losses but also slow down the iteration of your own AI business.

NVIDIA's most formidable barrier is not its hardware performance, but its CUDA software ecosystem, which it has cultivated for nearly two decades. Today, the vast majority of AI developers worldwide and leading AI algorithm libraries (such as the underlying operators of PyTorch) are deeply integrated with CUDA.

The biggest challenge large companies face with their self-developed chips is "easy to make, hard to use." Even if the chip successfully goes into production, if developers need months to rewrite low-level code to adapt to the chip’s compiler, the chip cannot be adopted internally.

To break the deadlock, the industry is launching a coordinated effort against CUDA. On one hand, major companies are actively developing their own compilers (such as Google’s XLA); on the other, OpenAI’s open-source programming language Triton is being widely anticipated. Triton aims to provide a higher level of abstraction than CUDA, allowing developers to write code once, with the Triton compiler translating it into machine code optimized for various underlying hardware, including Nvidia GPUs, AMD GPUs, and even proprietary ASICs. Once Triton or a similar intermediate-layer ecosystem matures, CUDA’s lock-in effect will be significantly weakened, and the migration cost for companies using their own custom chips will drop substantially.

The chip industry is a winner-takes-all market that heavily emphasizes economies of scale. NVIDIA can spread its massive R&D costs across millions of GPUs shipped globally, while large companies' in-house chips are typically used only internally or by their own cloud tenants, with shipment volumes often reaching only tens of thousands.

If sufficient scale cannot be achieved, the per-chip cost of in-house developed chips will far exceed that of directly purchasing general-purpose GPUs. Therefore, developing proprietary AI chips is inevitably a “game only a few giants can afford.” For mid- and long-tail tech companies, attempting to develop foundational AI chips is not only financially unsustainable but also technologically unable to keep pace with specialized chipmakers like NVIDIA. A wiser choice is to embrace existing cloud computing services.

Table 2: Comparison of advantages and disadvantages of commercial general-purpose GPUs versus proprietary ASIC chips from major tech companies

Analysis: The Ultimate Evolution of Future Computing Power Landscape

Based on the above analysis, "ME News Think Tank" makes the following assessment of the global AI computing landscape over the next 3-5 years:

NVIDIA will remain the indispensable leader in exploring the boundaries of frontier models for a very long time to come, because in the highly complex training of ultra-large-scale parameters, algorithms still exhibit significant uncertainty—a phase that critically requires the generality of GPUs and the error-correction capabilities and flexibility of the CUDA ecosystem.

However, during the inference phase after model maturity and in everyday deployments of large-scale internet applications—such as short video recommendations and search engine query rewriting—the high cost of general-purpose GPUs will compel companies to fully transition to proprietary, custom-designed ASICs. Future data centers will be heterogeneous: a small number of extremely expensive GPU clusters will be used for “alchemy” (training the next generation of large models), while vast arrays of proprietary ASIC clusters will handle billions of daily end-user requests.

Just as today’s large data centers design their own server motherboards and cooling systems, customizing at the chip level will become standard for top cloud service providers (CSPs). The ability to develop proprietary chips will be a core competitive advantage for CSPs when selling AI services to customers. Cloud providers without the capability to design underlying hardware will lose all margin potential in future price wars and become mere “data center landlords.”

In summary, major tech companies developing their own AI chips are not aiming to completely "kill" NVIDIA, but rather fighting for control over the distribution of profits in the AI era. By achieving computational independence on the inference side and within their core business operations, tech giants are reclaiming control over the technological lifeline and business profits. In this profound transformation of software-hardware synergy, computational power is no longer merely a purchasable commodity, but a company’s most critical strategic asset.

Source:

1. Semianalysis. (2024). AI Inference Economics: GPUs vs Custom Silicon.
2. Stanford University HAI. (2024). Artificial Intelligence Index Report 2024.
3. Bloomberg Technology. (2023). Microsoft to Debut AI Chip to Lessen Reliance on Nvidia.
4. Patterson, D., et al. (2021).Carbon Emissions and Large Neural Network Training. arXiv preprint.
5. AWS Official Blog. (2023). Amazon EC2 Inf2 Instances for Low-Cost, High-Performance Generative AI.