OpenAI's Jalapeño Chip Outperforms NVIDIA's Blackwell in Efficiency

Author: SemiAnalysis

Compiled by Deep潮 TechFlow

DeepChao Summary: Real-world test data for OpenAI’s first in-house inference chip, Jalapeño, has been revealed, showing energy efficiency that surpasses NVIDIA’s current flagship Blackwell and approaches the next-generation Rubin. For AI companies constrained by data center power bottlenecks, this chip could reshape the compute market. However, it remains uncertain whether this first-generation chip can truly challenge NVIDIA’s CUDA ecosystem.

Over the past two years, OpenAI has been quietly developing "Jalapeño," a reasoning chip that was just unveiled at Hot Chips. Rumors of a successful tape-out have been circulating for a while, but now we have the details. OpenAI invited us to examine the chip, enter their lab to verify its authenticity, and benchmark it using our InferenceX suite.

In June of this year, OpenAI announced a chip project in collaboration with Broadcom, designed from the ground up specifically for LLM inference. Design work began in mid-2024, and the entire process—from initial team hiring to manufacturing tape-out—took only about 16 months, an exceptionally fast ASIC development cycle.

First-generation chips typically lack competitiveness, but OpenAI has done the opposite—outperforming all NVIDIA, AMD, and Google chips we’ve tested across multiple top open-source models, establishing industry leadership. OpenAI achieved this through极致的软硬件协同设计. Surprisingly, rather than over-optimizing for any single aspect of model inference, OpenAI focused on building a general-purpose chip that delivers high performance across all scenarios.

This article will delve into the architectural details, software specifics, and performance results of Jalapeño on InferenceX.

Everyone says OpenAI’s chip is customized for OpenAI models, but that’s wrong—OpenAI built a general-purpose chip for AI inference.

The timeline is insane. This shows that the claim “using AI to accelerate chip design” is true. Despite the rapid timeline, OpenAI invested heavily, made pragmatic design decisions, and had a very strong team, so this isn’t surprising.

Looking at the specifications alone, it immediately becomes a strong contender:

Moreover, using HBM4 makes it competitive enough with NVIDIA and AMD's flagship GPUs:

Many media outlets have followed OpenAI’s offhand comments, claiming that the chip optimizes their models in ways other chips cannot. This is incorrect. Jalapeño is a general-purpose inference chip capable of running a variety of models and workloads, including our InferenceX benchmark—which we ran in the lab alongside OpenAI engineers. As a joke, OpenAI even showed us the chip running Doom, a game ported to their chip using only Codex prompts.

Below are our most important performance-per-watt results, measuring token throughput per megawatt of total power consumption. Jalapeño decisively outperforms all other chips. None of these results used multi-token prediction (MTP), while all other chips in the chart are configured at their optimal SKUs with MTP enabled.

Jalapeño outperforms Blackwell in performance per watt across nearly all scenarios, without any optimization for specific points on the curve. It excels not only in low-latency use cases but also in high-throughput scenarios. A more fair comparison is seen in single-token prediction results, where Jalapeño leaves every competitor far behind. In low-concurrency scenarios, Jalapeño demonstrates remarkable interactivity, achieving over 700 tokens per second per user on the DeepSeek R1 model with a concurrency level of 1.

Incredibly, all of this is achieved using single-token prediction (STP), without speculative decoding or prefill-decode separation. In addition to DeepSeek R1, we’ve also observed other models, including Kimi-K2.5 and GPT-OSS, achieving approximately 1,400 tokens per user per second. For all models, we confirmed that Jalapeño’s GSM8k evaluation results match those on NVIDIA chips.

Here are a few important notes. First, all data is provided by OpenAI. We personally tested InferenceX in our lab, but we did not run the full InferenceX benchmark suite, nor did we observe any AgentX results. AgentX is our preferred benchmark suite for comparing chip performance, as its dataset’s ultra-long context and multi-turn characteristics better reflect cache behavior under real production workloads. Frameworks that perform well on 8k1k may perform worse on AgentX, because real production workloads stress components such as routers, prefix caching mechanisms, cache management, and offload infrastructure—none of which are captured by single-turn 8k1k tests. For more details, please read our AgentX article.

AgentX - InferenceXv3: Can the CUDA moat still hold up in agent inference?

Second, we believe the comparison with Blackwell is incomplete and somewhat unfair. Jalapeño’s true competitors are chips like Rubin that also use HBM4. The Vera Rubin system has already begun shipping to customers, while OpenAI’s Jalapeño is still only available as engineering samples and remains some time away from mass production.

Therefore, performance should truly be compared to Rubin, not Blackwell. In a sense, we expected custom chips like Jalapeño to outperform Blackwell. As we noted in last month’s analysis of NVIDIA and CoreWeave’s performance claims, the Vera Rubin NVL72 delivers 5.4 times the performance per watt of the GB200 NVL72. We will later compare Jalapeño’s performance data to Vera Rubin’s figures from July.

Vera Rubin NVL72 vs. GB200 NVL72? Inference TCO and Architecture Analysis

Third, the models tested are not at the cutting edge of open-source development. NVIDIA and AMD have already published results for larger models, such as DeepSeek V4 Pro and Kimi K3, using AgentX. The larger and more recently released the model, the more complex it becomes to run it on new hardware. That said, the model OpenAI ran on Jalapeño is still substantial.

OpenAI’s design goal is performance per watt. The reason is simple: OpenAI is currently constrained by data center power, not budget or rack space, so the number of tokens per megawatt is critical. At Computex 2026, Jensen Huang stated that performance per watt, reliability, and long lifespan are core characteristics of future GPUs. He said verbatim: “If you have one gigawatt of power, then throughput per watt is revenue.” He also noted that it makes no sense to choose an inferior architecture simply because the chip is cheaper.

NVIDIA also emphasized this point in its Vera presentation at Hot Chips 2026, presenting the same revenue chart: “Today’s data centers are power-constrained.” Power is critical and drives revenue.

Operators cannot easily acquire additional megawatts. The time scales for adding GPUs and expanding grid capacity differ drastically. Data center power limits are constrained by numerous factors, such as utility interconnections, infrastructure, cooling capacity, and UPS/backup generator design. Grid delays frequently outpace hardware and construction timelines, driving demand for behind-the-meter power capacity—namely, installing gas turbines and on-site generators directly at data centers. This capacity lies beyond the utility meter and operates independently of the public grid, allowing operators to power facilities without waiting for grid interconnection or utility upgrades. This is precisely why xAI’s Colossus 2 relies heavily on behind-the-meter power, while its actual grid connection lags far behind. For more details, see our energy model.

As we wrote in our X post, tok/s/MW can be simplified to tokens per joule, since a watt is a joule per second. Therefore, tok/s/MW represents system efficiency and the ability to convert energy into tokens.

In this regard, Jalapeño outperforms even Rubin. OpenAI’s Jalapeño achieves higher token throughput per megawatt of STP than Vera Rubin’s MTP results, as reported by NVIDIA and CoreWeave in July. It also significantly surpasses the projected MTP results for the GB200 in 2025. As mentioned in our Vera Rubin article, the comparison between VR and the 2025 GB200 results is valid because both are in similar early-stage deployment phases. Comparing with the 2025 GB200 also maintains consistent software maturity levels. Following this logic, we compare three sets of results: Vera Rubin’s latest July 2026 figures, the GB200’s 2025 projections, and the current Jalapeño results. This comparison is highly reasonable, as these represent the best publicly available Rubin data. Moreover, OpenAI taped out its chip after Rubin. Both OpenAI and Rubin are still in immature stages, so performance is expected to continue improving.

In terms of performance and total cost of ownership (TCO), Vera Rubin and Jalapeño are evenly matched, producing nearly the same number of output tokens per dollar. However, as previously noted, Jalapeño’s results were achieved without speculative decoding, whereas Vera Rubin’s results utilized speculative decoding. Speculative decoding can reduce the cost per token by approximately 3 to 5 times. Once Jalapeño implements speculative decoding, its cost efficiency in delivering tokens will be even higher. Of course, some of the TCO advantage stems from avoiding NVIDIA’s high profit margins in favor of Broadcom’s lower—though still substantial—margins. But this is not the full story: for example, Meta and Microsoft’s AI ASIC projects have invested heavily for longer periods without achieving successful deployment. This suggests that cost is only one part of the equation. For a complete TCO breakdown of Jalapeño, see the SemiAnalysis AI Cloud TCO model.

Architecturally, OpenAI has chosen not to separate prefill and decode onto different chip pools. The draft and main models share the same chips and interconnect structure. This design philosophy sacrifices some theoretical efficiency in exchange for practical operational convenience. The motivation stems from the fact that workload composition changes over time—for example, the ratios of input, cache writes, cache reads, and output tokens have shifted significantly. This shift has occurred after we have gone through three model eras: knowledge, reasoning, and agents, as discussed in our recent article. Therefore, pre-allocating fixed quantities of heterogeneous prefill and decode silicon would lead to inefficiencies over time. OpenAI has opted for a homogeneous chip pool in this architecture and strives to ensure that chips perform well across all tasks.

And it did. On Kimi K2.5 (which underlies Cursor Composer 2.5), Jalapeño achieves nearly 700 tok/s/user—more than nine times faster than the second-best chip at 100 tok/s/user.

On GPT-OSS, it’s another overwhelming performance lead: Jalapeño’s interactive throughput per megawatt is nearly double the peak throughput of the GB200 and over 50 times higher than the GB200’s concurrency point of 1. Jalapeño points with higher concurrency utilize the EP8.

These results are impressive! However, we must be critical: they are based only on 8k1k, which is significantly easier to fine-tune, and they lack AgentX runtime data. As mentioned in our AgentX article, multi-turn and long-context workloads place greater strain on more aspects of the inference stack—such as routers and prefix caching. To excel in agent workloads, further optimizations are required. For more details, read the AgentX article.

AgentX - InferenceXv3: Can the CUDA moat still hold in agent inference?

All these results come from Jalapeño’s A0 stepping, just nine months after project launch. But the B0 stepping is now in production at the foundry! The optimizations in B0 deliver approximately 25% higher performance per watt compared to the earlier A0 silicon. Specifically, the B0 stepping achieves 13.4 PFLOPs of MXFP4 on a single reticle-sized compute chip, manufactured on TSMC’s N3P node. In comparison, a single Rubin compute chip achieves 17.5 PFLOPs of dense Rubin NVFP4 at a similar size and on the same node.

Given that Jalapeño has a TDP of only 700W, compared to Rubin’s 900–1150W per compute chip, this performance is even more impressive. Since Jalapeño is designed for inference rather than training, OpenAI had no need to increase the TDP to maximize FLOPs—this is understandable. Nevertheless, the results above demonstrate that Jalapeño delivers substantial peak theoretical FLOPs.

When directly compared to other accelerators, Jalapeño offers the highest HBM bandwidth per watt and the highest FLOPs per watt—performance on par with an 1800W Rubin Max-Q configuration.

Jalapeño will feature HBM4, making it one of the earliest chips to adopt this technology, ahead of existing TPU and Trainium projects. One of Jalapeño’s key architectural principles is maximizing HBM bandwidth, so compromising with non-top-tier HBM would contradict this goal. This results in a memory bandwidth of 15.4 TB/s per package, surpassing all other accelerators on the market using HBM3E. The 15.4 TB/s bandwidth suggests that Jalapeño’s HBM4 achieves a pin rate of 10 Gbps, slightly higher than Nvidia’s 9.6 Gbps in Rubin. HBM is likely supplied by Samsung.

OpenAI completed the Jalapeño tapeout in November 2025—more precisely, a CoWoS design tapeout, not just a top-level die. Within nine months after the November 2025 tapeout, and after only three months of debugging on real silicon, OpenAI has achieved excellent results with Jalapeño. This is even more impressive given that the team started from scratch on the software stack.

Meanwhile, Rubin’s CoWoS tape-out was completed in October 2025, a month earlier, but the only early results we’ve seen so far come from CoreWeave’s engineering samples. Nvidia has not allowed us to test and publish benchmarks, as OpenAI did, indicating that its chip software is still immature. Given OpenAI’s ability to rapidly deploy new models on its own chips, the CUDA moat may already be gone.

They are still far from optimized, and we can see that overall, Jalapeño delivered better data. We do not believe Nvidia’s hardware is inferior; rather, Jalapeño’s software debugging has progressed faster than Nvidia’s. This highlights the power of hardware/software co-design—the very area where top-tier frontier lab ASIC teams outpace more established commercial chip manufacturers. Counterintuitively, starting from scratch may also benefit OpenAI, as it allows them to make entirely new architectural decisions without being constrained by backward compatibility or legacy software versions.

Although OpenAI already has Jalapeño engineering samples, mass production is currently planned to ramp up gradually in 2027, with the majority of output scheduled for the end of next year. For more details on unit volumes and average selling prices, refer to the SemiAnalysis Accelerator Model.

OpenAI Jalapeno can be considered a truly large-scale ASIC.

Compared to Rubin’s timeline, Jalapeño is remarkably faster. As shown earlier, despite starting earlier, Rubin’s results are still outperformed by Jalapeño.

Jalapeño architecture

From an architectural perspective, the chip’s matrix engine uses the MXFP numerical format and a weight-stationary systolic array, similar to a TPU. However, unlike direct TPU comparisons, it supports smaller shapes/dimensions, meaning it avoids the abrupt performance cliffs that occur with matrix multiplications of mismatched shapes on larger systolic arrays.

It also features 64-bit scalar cores and FP32/INT32 vector cores. OpenAI has implemented redundancy at the tray level and built-in yield recovery at the core and channel levels. They claim that AI-assisted design reduced the SIMD area by 8% and the matrix engine area by 10%. Although they did not specify the exact process/voltage/temperature (PVT) conditions, they noted that the AI-assisted module improved timing and power consumption compared to the initial design.

The Jalapeño architecture is designed to eliminate memory movement of KVCache and weights, as well as fixed latency and overhead, enabling it to achieve closer-to-peak computational performance and bandwidth compared to other accelerators—even with small batch sizes or small tensor shapes.

Cores and HBM are divided into multiple slices, with each core slice having a low-latency local view of its own HBM slice. Synchronization between slices is achieved through a high-bandwidth dedicated collective network. This minimalist memory hierarchy gives Jalapeño significant potential advantages over GPUs, as GPU memory accesses must traverse complex memory systems, resulting in higher latencies that must be amortized or hidden across large tensor shapes.

This approach is feasible because careful placement of weights and KV limits synchronization between cores to a finite and known high-bandwidth communication, such as tensor parallel communication that can overlap with computation.

Additionally, there is an extra general-purpose NoC for general communication and access to the extended network. Overall, OpenAI achieves significant performance improvements and substantial power savings compared to Nvidia and Google through its simplified NoC and memory subsystem.

At its core, OpenAI describes an out-of-order (OoO) core with an L1 cache. This differs significantly from the pattern seen in other accelerators, which typically use software-managed scratchpads, often paired with asynchronous DMA support. The argument here is that this allows Jalapeño to avoid fixed overheads such as barrier latency, which in other accelerators like GPUs must be hidden or amortized by increasing the workload per core, making it harder to approach raw peak bandwidth or compute.

The cost is that Jalapeño relies on effective prefetching to ensure memory requests arrive in time, which is harder to predict and reason about. However, with Codex obtaining detailed trace information within a solid framework, finding the optimal kernel with the best prefetching for a given shape may require almost no human intervention. We believe this is precisely why OpenAI was able to rapidly deploy DeepSeek R1, Kimi K2.5, and GPT-OSS.

The core also supports smaller matrix dimensions, which—depending on how small—should make it more versatile across different models and batch sizes, and less sensitive to matrix dimension alignment, padding overhead, and inefficient tiling. For example, TPU, Trainium, and Etched chips feature very large systolic arrays that may require large batch sizes or model dimensions that are perfectly divisible to avoid inefficient tiling.

With Jalapeño, OpenAI focuses on eliminating fixed latency in the system to approach roofline performance as closely as possible across all regions of the Pareto curve. Theoretically, this could yield relative advantages over GPUs at multiple workloads:

The peak performance for low-latency/small-batch inference far exceeds that of GPUs. GPUs are limited by fixed overheads such as startup latency, barrier latency, and memory system latency.

Even with large batches or long contexts, there is potential to get closer to the hardware roofline.

This comes with a caveat: even if an upper performance limit exists in theory, actual kernels may be harder to achieve at that performance. Therefore, its approach appears to be:

Design maximum upper-bound performance for all workloads.

Let Codex handle the heavy lifting and find the kernel that achieves this limit.

The OpenAI team launched the InferenceX workload on Jalapeño with extremely fast turnaround times.

We are therefore optimistic about this approach.

If Jalapeño succeeds, it will send a strong signal.

The industry's obsession with programming models and perfect universal compilers will be broken by cutting-edge AI models.

OpenAI writes the Jalapeño kernel like assembly code.

Each kernel has manually optimized code, totaling approximately 3,000 lines.

Also supports correctness checks and custom sanitizers.

Early core operations were human-in-the-loop, not fully automated.

Later shifted to a more scaled-up internal version of Codex.

OpenAI plans to market this version to enterprise customers.

The internal service engine is called "Teacup".

Interestingly, OpenAI previously had no internal MLA kernel implementation.

Until they benchmarked DeepSeek using InferenceX.

Codex can generate usable, high-performance kernels so quickly without requiring intervention from the kernel engineering team.

This demonstrates the software pipeline's development capabilities.

OpenAI uses Gluon to program Jalapeño.

Gluon is OpenAI's core programming language.

Gluon is built on top of Triton and retains Triton's SPMD (Single Program, Multiple Data) programming model.

But it exposes the underlying programming abstraction.

For example, for NVIDIA GPUs, it provides an API that maps to PTX instructions.

Including MMA instructions, TMA instructions, and the mbarrier mechanism.

The most unique abstraction provided by Gluon is layout.

In general, a layout defines the mapping between hardware resources and tensor elements.

For example, register 5 of Warp 9 corresponds to the tensor element at row 6, column 7.

Gluon's layout abstraction is based on linear layouts.

This is a layout algebra invented by OpenAI.

Linear Layouts formally define what a layout is in mathematical terms.

And provide tools for the operational layout.

This supports many features, such as provably correct layout transformations.

Also includes optimal memory swizzling.

In Jalapeño's programming model, each Gluon program maps to a persistent thread.

We believe this suggests that Jalapeño is suitable for persistent kernel programming patterns.

Each program executes across multiple tiles, with work assigned by the programmer rather than a hardware scheduler.

OpenAI mentioned TensorInfo.

It is an abstraction of an explicitly encoded layout.

This may be a set of layouts designed for Jalapeño.

It will be powered by Linear Layouts.

Finally, each core provides data prefetching and decoupled out-of-order units.

For example, users can program to wait for prefetched data.

This data is locked by a semaphore.

Ironically, OpenAI models like GPT 5.6 Sol currently run on NVIDIA GPUs.

They are used to design chips that pose a real threat to CUDA's moat.

NVIDIA's own GPUs are actively boosting potential successors in real time.

When comparing across time, we can also see Jalapeño's development pace.

In less than two weeks, throughput for certain interaction scenarios increased by more than 2x.

Each tarball we receive from the Jalapeño team contains a wonderful world.

In addition to improved core performance, the Jalapeño team enabled TP32 within eight days.

Expand beyond the previous TP8 configuration to achieve a full rack-level setup for running large models.

This development speed is indeed impressive.

To verify performance before running on actual hardware, OpenAI also has a simulator called "chilisim".

Its accuracy is within 5% of real-world hardware, using a fixed-width trace bus.

Tracking capability is limited on A0 but significantly improved on B0.

This may be due to the actual operating data of the A0 silicon wafer.

The engineer demonstrated the Codex CLI running an internal model.

It is nicknamed "Raiku" or "5.3 Codex Spark," with a TPOT of 1.2ms.

The team also demonstrated a demo written by Codex running directly on the chip.

Including Doom at 36 FPS and FP32 fluid dynamics simulations.

Also includes "Liquid Light" mouse-drag visualization.

In terms of models, OpenAI's internal megakernel solution is nicknamed "gigakernel".

It is built around a single megakernel that loops on the device to reduce CPU overhead and startup time.

The team is further refining the calculation strategy during testing.

Internally, special attention is focused on coordinating the use of 1 million rollouts.

Previously, we mentioned that OpenAI did not use a prefill-decode separation on these chips.

This surprised us, as NVIDIA and AMD GPU performance has significantly benefited from PDD.

Even on homogeneous hardware.

Let’s dig deeper into why the Jalapeño team did this.

When the workload is fixed, separating prefill from decode appears attractive.

Prefill and decode impose different hardware workloads.

Assigning each stage to a separately tuned pool improves efficiency under the selected input-output ratio.

However, production traffic will not maintain this ratio.

The input-output sequence length, concurrency, cache hit rate, speculative acceptance rate, and latency targets all vary throughout the day.

Once devices are divided into prefill and decode pools, excessive prefill requests can leave decode chips idle while requests queue up.

However, too many decode requests have the opposite effect.

Operators must continuously predict the correct split and reserve backup capacity on both sides.

And rebalance a system whose ideal ratio is constantly changing.

In the unified system, certain resources may be underutilized during specific stages.

However, each device can still be used to serve the next request.

In a segregated system, an entire chip may remain idle simply because it belongs to an incorrect pool.

Local utilization appears better, but global utilization may be poor.

Separation will also disrupt locality.

The prefill worker generates a large KV cache that the decode worker needs immediately.

The system must transmit this status across the network for the process to continue.

This increases bandwidth consumption, synchronization, queuing, and another failure domain.

Cost also increases with the length of the input sequence, as the KV cache grows.

However, avoiding KV movement is primarily for power consumption and latency optimization.

Moving some KV can improve hardware utilization at the cost of increased power consumption and per-request latency.

Interchangeable clusters can shift capacity between latency-sensitive requests and throughput-oriented batches.

Fixed partitioning leaves hardware idle when traffic combinations change.

Additionally, changing the context length alters the balance between attention and FFN operations.

Any fixed hardware ratio is efficient only near the design point.

The same constraint applies to speculative decoding: the draft model must provide candidate tokens to the validator with extremely low latency. Separating the two into different dedicated pools transforms the tightly coupled decoding loop into a distributed protocol. Additional communication and coordination may consume the latency savings gained by the draft model. Only by placing both models on the same device and within a low-latency interconnect can the locality necessary to make speculative decoding worthwhile be preserved.

However, decoupling still prevails where demand is sufficiently large, stable, and predictable—especially when traditional GPUs require large, staged batches to achieve good throughput. But this is not a free lunch.

The Jalapeño system consists of CPU host racks and ASIC racks at the rack unit level. The host rack accommodates 16 host CPU trays named "Katsu." Each Katsu corresponds to one of the 16 ASIC trays on the right, named "Vindaloo." Each host is equipped with two Turin-class AMD EPYC CPUs, 1.5 TB of DRAM, 2x E1.S SSDs, and 2x M.2 SSDs per rack. Each tray also features a 400G (2x200G) front-end network. Each Katsu tray connects to each Vindaloo tray via eight external PCIe DAC cables, which are horizontally routed at the front of the rack. The system-level design was developed in collaboration with Celestica.

The ASIC rack, named "Chana," consists of 16 Vindaloo trays and 8 expansion switch trays (6 local + 2 global). Each Vindaloo tray contains 8 Jalapeño ASICs, totaling 128 Jalapeño ASICs per rack. The ASICs are connected to each Chana switch tray via copper backplane cabling, similar to Nvidia’s Oberon. The expansion topology is divided into local and global domains. The local domain connects the 128 ASICs within the rack. The global domain connects up to 16 racks, or 2,048 ASICs. We will explain bandwidth and topology in more detail below.

The side-mounted host rack consumes approximately 50 kW (31 kW during production), while the ASIC rack consumes 130 kW. The entire dual-rack system totals approximately 160 kW, which is roughly equivalent to the power consumption of a dual-width GB300 rack.

OpenAI can connect up to 2,048 Jalapeño XPUs within a single scaling network. The scaling network consists of two domains. The local domain connects all 128 XPUs within a rack via a backplane. The global domain uses a hybrid of copper and optical interconnects to link the 2,048 XPUs across 16 racks. Each rack contains eight Chana switch trays. Six central Chana switches serve the local domain, each equipped with a 102.4T Tomahawk 6 switching ASIC. The top and bottom two Chana switches serve the global domain; we believe these may be two 102.4T Tomahawk 6 switches per switch tray, enabling up to 204.8T per tray.

In this local domain, 128 Jalapeño chips provide a unidirectional bandwidth of 4.8 Tb/s per XPU. They are fully mesh-connected to six 102.4 Tb/s Tomahawk 6 ASICs, equivalent to 48 differential pair (DP) male-female connectors per XPU. In total, 6,144 DP passive copper cables are used for local expansion per rack.

In the global domain, 2,048 XPUs across 16 racks are interconnected via a copper backplane, electrical 204.8T TH6 switches, 1.6T optical modules, and optical circuit switches. Each XPU has a unidirectional global link bandwidth of 1.6 Tb/s, featuring 16 differential pair (DP) male-female connectors on the backplane for connection to the global switch. Each global switch tray contains two ASICs, with their egress bandwidth distributed between the backplane and front-panel optics.

Between the local and global domains, there are 64 DP pairs per rack connector per XPU, totaling 8,192 DP pairs of passive copper cables per rack.

The global domain employs a pure rail architecture composed of eight rails within the global domain. We believe OpenAI routes optical links within the global domain using optical circuit switches (OCS) installed on each rack. Each XPU’s 1.6 Tb/s global bandwidth reaches the global switch tray via a copper backplane, then exits the switch through a 1.6 T optical module on the front panel. It first passes through a passive optical switch before leaving the rack. This enables scaling the domain to 2,048 XPUs, organized across 16 racks, with 128 XPUs per rack.

Since the expanded network accounts for only about 10% of the total system cost, this flexibility provides valuable optionality for future models with 10 to 20 trillion parameters or context windows of 2 million to 4 million tokens. For deployment, OpenAI is collaborating with neocloud. It has also been gathering reliability data with data center partners over the past month while optimizing deployment time from dock to rack.

Next, we discuss Jalapeño’s future, as its first production token is imminent. The next goal is 100 MW, with the main challenges lying in hardware: how much they can produce, how effectively they can deploy and operate data centers, and how they handle monitoring and resilience. Software has already been validated, and with internal models in place, any software first-mover advantage is easily replicable. Behind the paywall, we’ll explore the impact on chip companies such as NVIDIA, AMD, and Cerebras, which have secured multi-year contracts with OpenAI.

We also cover the production volume, quantity, and timeline of next-generation chips in the Accelerator model.

The latter part is paid content and has not been further translated due to access restrictions; however, the preceding text already provides substantial information on OpenAI’s challenges to NVIDIA’s chips. For investors interested in the semiconductor sector and NVIDIA’s stock price, this information offers fresh and reliable insights.