Author: Xiao Bing
AMD acquires Taalas: inference chips begin embedding model weights directly into silicon
On August 6, AMD announced the acquisition of Toronto-based chip company Taalas, with the transaction price undisclosed. This startup, founded in 2023 and having raised a total of funding of $219 million, did something that sounds a bit crazy: it burned the weights of an AI model directly into the metal layers of a chip to create a dedicated chip capable of running only one model.
GPUs work by storing model parameters in high-bandwidth memory (HBM) and repeatedly moving data in and out of computational units during inference. This "movement" consumes significant energy and time, a bottleneck known in the industry as the "memory wall." Taalas eliminates this movement entirely by embedding weights directly into the chip’s transistors, integrating storage and computation into a single unit.
The cost is that this chip can only run one model, but the benefit is a orders-of-magnitude improvement in speed and energy efficiency.
What does 17,000 tokens per second mean?
Taala's technical verification chip, HC1, is built on TSMC's 6nm process, features a die size of 815 square millimeters, and contains 53 billion transistors. It integrates Meta's Llama 3.1 8B model.
HC1 benchmark data: approximately 17,000 tokens/second per user. For comparison, the Nvidia H200 achieves about 230 tokens/second and the H100 about 150 tokens/second on the same model—73 times faster, with only one-tenth the power consumption.
A clearer economic breakdown: Taalas reports an inference cost of approximately $0.0075 per million tokens (Llama 8B), while GPU solutions range from $0.20 to $0.49. Each HC1 card consumes 250W; a standard air-cooled rack housing 10 cards requires only 12–15 kW, with no liquid cooling needed, whereas GPU racks typically consume 120–600 kW.
Forbes analyst Karl Freund evaluated in February: “10 times faster than the fastest inference platform (Cerebras Wafer-Scale Engine) and two orders of magnitude faster than GPUs.”
However, there are several important caveats. HC1 uses Taalas’s proprietary 3-bit data format paired with 6-bit parameters, resulting in aggressive quantization with demonstrable quality degradation on complex reasoning tasks. The 17,000 tokens/second figure comes from a vendor benchmark using 1K input/1K output and does not reflect real-world production performance. Moreover, Llama 3.1 8B is a model released in mid-2024, and its 8B-parameter quantized version can even run on a Raspberry Pi 5. HC1 demonstrates architectural potential, not the competitiveness of a mature product.
What should I do if the model is upgraded?
Flashing the model into the chip raises the most intuitive question: what if the model is updated? Does the chip become obsolete?
Taalas's response: No, it won't become obsolete. Traditional ASIC design cycles take over two years. Taalas has developed an automated design process that allows new models to be compiled into new chips by modifying only a small amount of metal masking at the chip's top level, reducing the cycle to about eight weeks. The company's cost model includes provisions for three chip updates over a four-year lifecycle.
This answer can alleviate some of the anxiety, but not eliminate it entirely.
The flexibility of GPUs lies in the fact that the same card can run Llama today, Claude tomorrow, and a new, unreleased model the day after. Taalas’s chips cannot do this. If a customer needs to serve multiple models simultaneously—which is extremely common in production environments—they would need a different chip for each model, increasing complexity in inventory management and operations.
HC2 is under development, targeting a model with approximately 20 billion parameters and improved precision using 4-bit floating point. Taalas originally planned to support state-of-the-art models by the end of 2026.
AMD's acquisition has enabled synergy between the Instinct GPU product line and the Helios rack system, allowing customers to use GPUs for dynamic, variable workloads and Taalas chips for stable, high-frequency single-model inference.
The arms race in inference chips
AMD's acquisition of Taalas is the latest in a wave of inference chip acquisitions over the past eight months.
In December 2025, Nvidia acquired Groq for $20 billion, securing its SRAM-based low-latency inference architecture.
In May 2026, Cerebras went public with a market capitalization of approximately $56 billion, becoming the largest tech IPO since Snowflake in 2020.
In June, Etched ended its more than two-year stealth period, announcing that its first Transformer-specific chip had successfully taped out on TSMC’s N4P process, with over $1 billion in customer contracts secured. Intel is reportedly in advanced talks to sign a letter of intent to acquire SambaNova, while Qualcomm is engaging with Tenstorrent.
These transactions point to the same conclusion: reasoning is becoming the main battleground for AI compute spending.
Industry forecasts predict that by 2026, inference will account for two-thirds of AI computing spending, and by 2030, dedicated inference ASICs could capture 45% of the inference market. Nvidia’s GPUs still dominate the training segment, but in inference, its general-purpose architecture is facing mounting challenges from specialized chips across multiple fronts.
Taalas sits at the most extreme end of this spectrum: it abandons even the generality of a "class of models" and instead builds a dedicated chip for a "single model."
Groq bypasses the memory wall by replacing HBM with SRAM, Cerebras brute-forces its way through using wafer-scale area, Etched optimizes solely for the Transformer architecture, and Taalas makes an even bolder bet: that AI models will eventually stabilize, much like communication protocols converge to a few standards. When models no longer change monthly, manufacturing dedicated chips for just the most popular few becomes economically viable.
The betting model will "freeze"
Vamsi Boppana, Senior Vice President of AMD, stated in the announcement that the acquisition aims to provide customers with "the optimal computing solution for every AI workload." This translates to a competitive strategy: GPUs handle flexibility, while Taalas chips deliver maximum efficiency, allowing customers to combine them as needed.
Whether this logic holds depends on one core assumption: whether the update cycle of AI models will slow down.
If large models continue to receive architectural updates every few months, embedding weights into silicon is like pouring concrete into quicksand—the chip won’t even break even before the model becomes obsolete. But if model capabilities begin to converge and top models routinely provide stable service for one to two years, then specialized chips like Taalas’s will become as commonplace as baseband chips in the telecommunications industry, evolving from wild experiments into obvious choices.
Looking back over the past 12 months, the pace of model updates has indeed undergone subtle changes. The intervals between Llama series updates—from 3.1 to 3.2 to 4—have lengthened, while the architectural shifts between GPT versions—from 4 to 4o to 5—have narrowed. More new models are now being developed through distillation, quantization, and fine-tuning of existing architectures, indicating a slowdown in base model iteration.
If this trend continues, Taala has bet correctly.
Twitter: https://twitter.com/BitpushNewsCN
BitPush Telegram community: https://t.me/BitPushCommunity
BitPush TG subscription: https://t.me/bitpush