Google Launches New AI Chips Focused on Privacy and High Memory

icon币界网
Share
AI summary iconSummary
Google launched two new AI chips, the TPU 8tx and TPU 8i Max, highlighting privacy and high memory capacity. The TPU 8tx employs confidential computing to safeguard data, while the TPU 8i Max doubles memory throughput. Google also open-sourced TPU Sync to reduce latency. This move comes as on-chain data indicates growing interest in AI infrastructure, coinciding with a shift in market sentiment reflected by the Fear & Greed Index. The TPU product line now serves over 100 clients, including universities and AI labs. Google aims to challenge NVIDIA, whose stock declined following CEO-related issues.
CoinDesk reports:

On Monday, Google unveiled two sets of AI computing systems, aiming to challenge NVIDIA with chips focused on privacy protection and large memory capacity.

With the addition of the TPU v8tx and TPU v8i Max to Google’s eighth-generation Tensor Processing Unit product line, enterprises will now have four configurations available for training and running AI models. Google states that its customer base has reached nearly 100 TPU clients, including AI labs and universities.

Unlike NVIDIA’s widely used graphics processors, TPUs are accelerators specifically designed for AI. Google originally developed these chips for its own services, but now increasingly licenses them to external companies, including customers deploying them on-premises.

Amin Vahdat, Google’s Chief Technology Officer and Senior Vice President of AI and Infrastructure, wrote in Google’s announcement: “As computing shifts from human scale to agent scale, enterprises are seeking accelerators capable of reasoning across the vast context windows of AI models.”

Protect the privacy of sensitive information

TPU 8tx addresses a key challenge faced by many serious AI adopters: how to protect sensitive information when models use data.

Data is encrypted during storage and transmission, but typically needs to be decrypted in memory for processing. Google states that this configuration uses hardware-level confidential computing to isolate customer data and proprietary model parameters even during runtime.

The system combines the TPU v8t chip with fifth-generation AMD EPYC processors and Google’s low-latency Boardfly network. Google states that, compared to the TPU v8t, this system can reduce latency by up to 50% with negligible performance impact from additional protections.

In other words, tasks requiring privacy protection do not compromise speed or accuracy.

Enterprises can fine-tune and run models on the same system without transferring sensitive information between clusters. However, confidential computing cannot prevent poorly designed AI agents from misusing data they are authorized to access.

Break through AI's memory bottleneck

The TPU 8i Max addresses another limitation: memory. Autonomous agents require access to significantly more data, much faster than traditional AI tasks, when analyzing content such as data, codebases, documents, or hours of video.

Google has packaged two TPU 8i chips into a single module, doubling the computational power, on-chip SRAM, and high-bandwidth memory throughput compared to the TPU 8i. This design packs twice the silicon into the same rack footprint.

Vahdat wrote: "Our fourth system, the TPU 8i Max, was created to address another issue: the memory wall."

Google has also open-sourced TPU Sync, a software that moves AI workloads between chips, servers, and storage. Google claims the system can reduce certain data transfer latencies and response startup times by more than 90%.

As Google released the above statement, NVIDIA is facing increasing pressure from Wall Street. On Thursday, NVIDIA’s stock fell 9.3%, and it dropped another 5% in Monday trading after CEO Jensen Huang acknowledged an extramarital affair with a woman suspected by U.S. intelligence agencies of engaging in espionage for China. NVIDIA’s board has initiated an independent investigation, though the company stated there is no evidence that corporate secrets have been compromised.

This scandal has intensified external concerns about NVIDIA’s growth prospects, as competitors aim to challenge its dominance in AI chips. Reports suggest that OpenAI is planning to purchase Google’s TPUs to reduce its reliance on NVIDIA. While this information has not been confirmed, Google’s expanding lineup of chips could grant AI developers greater bargaining power and threaten NVIDIA’s market share and future growth expectations.

Google competes not only on chip speed but also on specialized hardware and software. Its open-source TorchTPU project also enables developers to run PyTorch and Hugging Face models on TPUs without rewriting their applications.

Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.