On Monday, Google unveiled two sets of AI computing systems, aiming to challenge NVIDIA with chips focused on privacy protection and large memory capacity.
With the addition of the TPU v8tx and TPU v8i Max to Google’s eighth-generation Tensor Processing Unit product line, enterprises will now have four configurations available for training and running AI models. Google states that its customer base has reached nearly 100 TPU clients, including AI labs and universities.
Unlike NVIDIA’s widely used graphics processors, TPUs are accelerators specifically designed for AI. Google originally developed these chips for its own services, but now increasingly licenses them to external companies, including customers deploying them on-premises.
Amin Vahdat, Google’s Chief Technology Officer and Senior Vice President of AI and Infrastructure, wrote in Google’s announcement: “As computing shifts from human scale to agent scale, enterprises are seeking accelerators capable of reasoning across the vast context windows of AI models.”
Protect the privacy of sensitive information
TPU 8tx addresses a key challenge faced by many serious AI adopters: how to protect sensitive information when models use data.
Data is encrypted during storage and transmission, but typically needs to be decrypted in memory for processing. Google states that this configuration uses hardware-level confidential computing to isolate customer data and proprietary model parameters even during runtime.
The system combines the TPU v8t chip with fifth-generation AMD EPYC processors and Google’s low-latency Boardfly network. Google states that, compared to the TPU v8t, this system can reduce latency by up to 50% with negligible performance impact from additional protections.
In other words, tasks requiring privacy protection do not compromise speed or accuracy.
Enterprises can fine-tune and run models on the same system without transferring sensitive information between clusters. However, confidential computing cannot prevent poorly designed AI agents from misusing data they are authorized to access.
Break through AI's memory bottleneck
The TPU 8i Max addresses another limitation: memory. Autonomous agents require access to significantly more data, much faster than traditional AI tasks, when analyzing content such as data, codebases, documents, or hours of video.
Google has packaged two TPU 8i chips into a single module, doubling the computational power, on-chip SRAM, and high-bandwidth memory throughput compared to the TPU 8i. This design packs twice the silicon into the same rack footprint.
Vahdat wrote: "Our fourth system, the TPU 8i Max, was created to address another issue: the memory wall."
Google has also open-sourced TPU Sync, a software that moves AI workloads between chips, servers, and storage. Google claims the system can reduce certain data transfer latencies and response startup times by more than 90%.
As Google released the above statement, NVIDIA is facing increasing pressure from Wall Street. On Thursday, NVIDIA’s stock fell 9.3%, and it dropped another 5% in Monday trading after CEO Jensen Huang acknowledged an extramarital affair with a woman suspected by U.S. intelligence agencies of engaging in espionage for China. NVIDIA’s board has initiated an independent investigation, though the company stated there is no evidence that corporate secrets have been compromised.
This scandal has intensified external concerns about NVIDIA’s growth prospects, as competitors aim to challenge its dominance in AI chips. Reports suggest that OpenAI is planning to purchase Google’s TPUs to reduce its reliance on NVIDIA. While this information has not been confirmed, Google’s expanding lineup of chips could grant AI developers greater bargaining power and threaten NVIDIA’s market share and future growth expectations.

Google competes not only on chip speed but also on specialized hardware and software. Its open-source TorchTPU project also enables developers to run PyTorch and Hugging Face models on TPUs without rewriting their applications.

