OpenAI's Jalapeño Chip Outperforms NVIDIA Blackwell in Key Metrics

Original author: Dong Jing

Source: Wall Street Journal

OpenAI's first in-house chip, Jalapeño, has emerged, dramatically disrupting the existing AI chip landscape—not just a product launch, but a signal that AI is reshaping how chips are designed.

According to Wall Street Journal article, Bloomberg reported on August 25 that OpenAI stated Jalapeño leads NVIDIA’s GB300 in two key metrics: AI workload processed per unit of power consumption and response speed. The chip, developed in collaboration between OpenAI and Broadcom, is specifically designed for AI inference and is expected to enter practical use as early as later this year. Richard Ho, Head of Chips at OpenAI, said Jalapeño delivers strong performance at just 700 watts of low power consumption, significantly reducing data center electricity costs.

According to semiconductor research firm SemiAnalysis, the chip took only about 16 months from design initiation to tape-out, with the critical CoWoS packaging tape-out completed in November 2025—just nine months ago—far below the industry’s typical 18- to 36-month cycle.

SemiAnalysis researchers visited the OpenAI lab in person and tested Jalapeño using their proprietary benchmark suite, InferenceX, concluding directly that the chip outperforms all NVIDIA, AMD, and Google chips they have previously tested on a per-watt performance (perf/W) basis.

More notably, this performance was achieved without enabling optimizations such as speculative decoding or separating prefilling from decoding, which Jalapeño has not yet implemented, while all other chips being compared were already running their optimal configurations.

No wonder even renowned semiconductor analyst Dylan Patel bluntly said, “It’s not just Blackwell that has been overtaken—NVIDIA’s Rubin chip has been surpassed as well!”

Analysis suggests that this outcome directly challenges NVIDIA's market position and prompts the market to reassess the competitive landscape of AI chips.

SemiAnalysis's test results show that Jalapeño has a significantly greater advantage in inference efficiency.

On the GPT-OSS 120B model, Jalapeño outputs approximately 1,459 tokens per second, while the NVIDIA GB200 manages only 535; in terms of end-to-end latency, Jalapeño completes a task in just 1.65 seconds, whereas the GB300 requires nearly 6 seconds—a 3.6x difference. In high-interaction scenarios, when the GB300 is pushed to its maximum decoding speed of 169 tokens per second, Jalapeño achieves 104.3 times the throughput.

On the core metric of tokens output per megawatt per second, Jalapeño achieves approximately 53 million tokens/MW/s on the GPT-OSS model, while the GB200 NVL72 achieves only about 10 million.

SemiAnalysis points out that this metric is essentially equivalent to the number of tokens produced per joule, directly determining the revenue ceiling of a data center—under current conditions where computing power is constrained by electricity, this advantage is particularly critical.

From a system-level cost perspective, according to SemiAnalysis, which includes power, cooling, and networking in its calculations, the total cost of ownership per chip per hour for Jalapeño is approximately $1.56, nearly identical to H100’s $1.55, while NVIDIA’s Vera Rubin reaches $3.61.

It is worth noting that SemiAnalysis also raised several important reservations.

First, the model used in the test is not the most cutting-edge available; NVIDIA and AMD have already published results on larger models such as DeepSeek V4 Pro and Kimi K3 using the AgentX suite, while Jalapeño has not yet completed AgentX testing—this suite better reflects performance in real-world production scenarios involving multi-turn interactions and long contexts.

Second, a more fair comparison would be NVIDIA’s Vera Rubin, which also uses HBM4, rather than Blackwell; Vera Rubin offers approximately 5.4 times better performance per watt than the GB200 NVL72 and has nearly identical total cost of ownership (TCO) per token compared to Jalapeño.

Third, Jalapeño is still in the engineering sample phase, with mass production expected to gradually ramp up by 2027.

The deep integration of AI tools is a primary reason Jalapeño was able to complete development at such an astonishing pace. According to SemiAnalysis, OpenAI extensively used its internal AI models during the chip design process, including the widely watched GPT-Astra.

Social media platform X user Andrew Curran cited OpenAI information indicating that GPT-Astra was deeply involved throughout the entire development of Jalapeño:

The team leveraged Codex and GPT-Astra to fine-tune three open-source models, originally not included in Jalapeño’s production roadmap, to high-performance status within two months.

This means that AI is not only accelerating the chip design process itself but also rapidly expanding the range of models that chips can support.

SemiAnalysis's data further quantifies this contribution:

AI-assisted design reduced the area of the SIMD unit by 8% and the matrix engine by 10%, while also delivering improved timing and power performance compared to the initial version.

At the software level, OpenAI used an internal extended version of Codex to write the Jalapeño kernel, with some kernel code reaching approximately 3,000 lines; for the most performance-intensive attention mechanism and MoE modules, the AI-generated code is 1.5 to 1.8 times faster than implementations by top human engineers.

SemiAnalysis offered a sharp assessment: OpenAI models running on NVIDIA GPUs, such as GPT-5.6 Sol, are being used to design a chip that poses a real threat to CUDA’s moat—“NVIDIA’s own GPUs are actively nurturing their potential successor in real time.”

While the general public assumes Jalapeño is a chip specifically customized for OpenAI’s own models, SemiAnalysis’s conclusion is the opposite: Jalapeño is a general-purpose chip designed for AI inference, capable of running various models and workloads, including the game Doom ported using Codex.

At the architectural level, Jalapeño’s core design philosophy is "eliminating fixed latency." Unlike GPUs that rely on complex memory hierarchies, Jalapeño directly binds compute cores to HBM slices, with cores synchronized via a dedicated high-bandwidth collective network, significantly reducing memory access latency.

In addition, Jalapeño employs an out-of-order (OoO) core with L1 cache, rather than the software-managed scratchpads commonly used by other accelerators, enabling it to come closer to hardware theoretical peak performance in low-latency, small-batch scenarios.

In terms of memory bandwidth, Jalapeño uses HBM4 to achieve a single-package memory bandwidth of 15.4 TB/s, with 22 GB/s per watt of HBM bandwidth, compared to NVIDIA Rubin at 11.1 and GB300 at just 5.71.

SemiAnalysis notes that Jalapeño's HBM4 pin speed reaches 10 Gbps, slightly higher than NVIDIA Rubin's 9.6 Gbps, with Samsung likely as the HBM supplier.

It is worth noting that Jalapeño chose not to implement prompt and decode separation deployment (PDD). SemiAnalysis provided a detailed explanation:

In a real production environment, parameters such as input-output ratios, concurrency levels, and cache hit rates continuously fluctuate; fixed pooling leads to reduced overall utilization, whereas homogeneous resource pools can dynamically allocate resources in response to traffic changes, balancing latency-sensitive requests and high-throughput batch processing.

SemiAnalysis presented a significant assertion in its report: CUDA's moat may already be dead.

This is based on a comparison of software startup speeds. NVIDIA's Rubin CoWoS tapeout was completed one month earlier than Jalapeño, but so far, the only publicly available Rubin benchmark data has come from CoreWeave's engineering samples; NVIDIA has not opened its labs to third parties for free testing, as OpenAI has. SemiAnalysis believes this reflects a gap in software maturity, not hardware performance.

Building the software stack from scratch allowed OpenAI to shed historical baggage and make cleaner architectural decisions. OpenAI uses its proprietary kernel programming language, Gluon (built on Triton), and an internal inference engine called "Teacup," leveraging Codex to rapidly optimize kernel performance. SemiAnalysis observed that within less than two weeks, Jalapeño achieved more than a 2x increase in throughput at specific interaction speeds; within eight days, the team scaled tensor parallelism from TP8 to TP32, enabling full-rack deployment across multiple racks.

However, SemiAnalysis also explicitly noted that the current tests only cover the relatively simple 8k1k workload and have not yet completed multi-round, long-context testing for AgentX. Under more complex agent workloads, the performance of components such as the router and prefix caching will become new challenges.

The story of Jalapeño is far from over.

According to SemiAnalysis, the chips currently used in public testing are all A0 stepping samples, while B0 stepping has entered the foundry and is expected to deliver approximately 25% better performance per watt than A0—the B0 single compute die will achieve 13.4 PFLOPs of MXFP4 compute power with a TDP maintained at 700W.

At the system level, OpenAI partnered with Celestica to design a complete rack solution: each ASIC cabinet contains 128 Jalapeño chips, and the dual-cabinet system consumes approximately 160 kW, comparable to NVIDIA’s GB300 dual-width cabinet.

In terms of scalable deployment, a single expanded network domain can connect up to 2,048 Jalapeño XPUs across 16 racks. In terms of mass production, SemiAnalysis expects gradual ramp-up beginning in 2027, with the next milestone target being a 100 MW deployment scale.

The broader strategic picture is that OpenAI has signed a 10GW custom accelerator agreement with Broadcom, a scale roughly equivalent to the full generating capacity of ten nuclear power plants.

Wall Street Journal article reports that Richard Ho, OpenAI’s chip lead, said the company has reached a level of power consumption and cost that can substantially reduce infrastructure expenses, "This is just the first step." He also emphasized that NVIDIA remains a key partner, as OpenAI’s demand for computing power is enormous and it will not abandon its current suppliers in the short term.

Analysts point out that Jalapeño’s significance may not lie in how many NVIDIA chips it can replace today, but in proving something previously widely doubted: that an AI company, using AI tools, can design a truly competitive chip in record time. This flywheel has already begun to turn.