[Guide] The reasoning speed of GPT-5.6 reaches an astonishing 750 tokens per second! Industry experts reveal it will operate across 100 wafers. Is AI transitioning from thinking to instant response—is the era of real-time intelligence truly here?
According to various reports, GPT-5.6 is about to be opened to the public.
Recently, various speculations about this model have gone viral on X.
On June 26, OpenAI officially announced the new GPT-5.6 family.
Moreover, the official blog states: OpenAI plans to launch its next-generation model, GPT-5.6 Sol, this month on custom hardware from chip giant Cerebras, achieving a staggering inference speed of 750 tokens per second!
In other words, complex Agent operations that previously required waiting several minutes can now be completed in the blink of an eye.
Clearly, OpenAI has taken a groundbreaking step in co-designing hardware and models.
With the recent unveiling of Jalapeño, the industry's first self-developed AI inference chip, we can sense that OpenAI has ambitions to become a full-stack AI empire.
In martial arts, speed is unbeatable: a 750 tokens/s advantage.
What does "750 tokens per second" mean?
For humans, this is equivalent to reading and outputting about 500 to 600 Chinese characters in one second.
The text you're seeing, GPT-5.6 Sol, can generate in less than a fraction of a second.
On X, renowned developer Caleb Shepherd excitedly said: “This is the most exciting thing to me—GPT-5.6 Sol running on Cerebras. It’s not just that coding speed has increased, but the pace at which computers operate has undergone a qualitative leap. We no longer have to wait two minutes for an AI to click a button.”
For a long time, despite large models becoming increasingly intelligent, "inference latency" has remained the biggest bottleneck in deploying real-time, multi-step agent tasks.
When models grow to trillions of parameters, traditional GPU clusters often encounter physical bottlenecks in inter-node communication (NVLink interconnects).
OpenAI's response is: Don't make the model adapt to the hardware; make the hardware and model integrate as one.
According to preliminary information disclosed by the official, GPT-5.6 Sol will be made available to select customers on a very limited scale in July, with broader rollout occurring gradually as capacity increases.
As many have speculated online, this is undoubtedly an extremely expensive service, a privileged offering tailored for top-tier businesses willing to pay for speed.
How can a 3-trillion-parameter behemoth fit into a chip?
When the news of 750 tokens/s emerged, Peter Gostev, head of LLM Arena, posed a question that puzzled everyone:
What’s really going on with GPT-5.6 Sol on Cerebras? As far as I understand, this appears to be the full, uncut version of the model—including multimodal capabilities such as vision—rather than a stripped-down, crippled version like the previous GPT-5.3-Codex-Spark, which lacked vision and context.
But my understanding is that Cerebras’ single chip can accommodate at most a 700 to 900 billion parameter model. So, has the model been reduced in size, or is there a new type of chip I’m unaware of, or perhaps some new multi-chip coordination technology?
This question immediately sparked extensive discussion among netizens.
Some jokingly referred to this as conducting a forensic-level chip audit at midnight, saying, "If this is truly the same complete model, it’s like someone managed to squeeze an entire superyacht into a glass bottle without telling you how."
Soon, seasoned technical expert Bleys Goodson provided a highly compelling, in-depth analysis—
GPT-5.6 Sol isn't fitted with a single chip—it spans 70 to 100 Cerebras wafer-scale chips!
Ultimate deployment aesthetics: “One wafer, one network”
Industry experts estimate that the specifications of GPT-5.6 Sol are extremely large:
- Total parameters: approximately 3 trillion
- Activation parameter: approximately 150 billion
- Number of network layers: Approximately 70 to 90 layers
To achieve optimal reasoning service performance, OpenAI and Cerebras adopted an extraordinarily lavish and impressive deployment approach—deploying each layer of the neural network on an entire Cerebras wafer.
As one netizen pointed out, by increasing the number of pipeline stages, you can theoretically scale to models of any size, as long as you have enough wafers to link them together; this will not affect token generation speed, and may only slightly impact the time to first token (TTFT).
Architectural Restructuring Like Cutting Off a Wounded Arm—A Necessity-Driven Lightweight KV Cache
However, having massive amounts of wafers alone is not enough. A key feature of the Cerebras chip architecture is its enormous on-chip SRAM (static random-access memory), which is extremely fast but very limited in capacity.
If OpenAI were to use traditional heavy KV caching in GPT-5.6 Sol, as it has in the past, this expensive SRAM bandwidth would be instantly exhausted.
This leads to the most core strategic shift of this collaboration: reconstructing the model around specific hardware.
Bleys Goodson noted that since OpenAI is deeply involved in hardware co-design, they most likely abandoned traditional attention mechanism caching schemes in favor of more advanced lightweight designs.
The most likely scenarios include:
Architecture similar to DeepSeekV4: Highly optimized cache usage.
Hybrid SSM Design: Combine linear-time complexity models like Mamba with Transformers to completely eliminate the historical burden of KV Cache.
Additionally, renowned developer John Lam proposed an intriguing hypothesis: decoupling attention from the FFN.
He speculated that OpenAI may be using traditional GPUs to handle attention computations, while employing vast numbers of Cerebras wafers to brute-force the feedforward neural network computations.
This is no baseless claim. Netizens quickly uncovered Cerebras’s previous blog details regarding the deployment of Kimi K2.6:
Cerebras stores the raw weights of Kimi K2.6 at 4-bit precision on the CS-3 system while performing computations at 16-bit floating-point to maintain accuracy. Weights are distributed across multiple wafers, and activation values are streamed between wafers. Fully connected communication between layers relies entirely on the on-wafer network architecture, which offers more than 200 times the bandwidth of NVLink on Nvidia’s NVL72. Combined with custom operators and speculative decoding, they achieve inference speeds of nearly 1,000 tokens per second on trillion-parameter MoE models.
Official specifications show that the revolutionary CS-3 system not only offers unmatched speed but also easily scales to models with up to 24 trillion parameters on a single logical device!
As someone marveled: “If this is truly the full version of Sol running on Cerebras, then the previously assumed ceiling for model sizes has been shattered tonight.”
The real trump card—OpenAI debuts its self-developed chip, 'Jalapeño'
Just prior to this, OpenAI officially launched its first ever self-developed chip—Jalapeño.
The release of this chip directly reveals the deeper logic behind OpenAI’s collaboration with Cerebras: through exploring third-party leading inference hardware, OpenAI fully understood the critical limitations and value of dedicated inference architectures, and transformed this insight into its own controllable foundational platform.
Jalapeño is one of the mildest varieties of Mexican chili peppers. OpenAI’s use of the name clearly suggests: this is just an appetizer.
This chip is a custom ASIC designed specifically for large model inference. From the first line drawn, every transistor is optimized for one purpose: running large models.
Surprisingly, Jalapeño not only runs on OpenAI’s own models but also supports LLMs across the entire industry, demonstrating significant platform ambitions.
Moreover, the design and fabrication of this chip took only nine months.
Behind this is an extremely powerful industry alliance:
Architecture-led: OpenAI personally designed the underlying architecture.
Chip Implementation and Interconnection: Chip giant Broadcom provides robust implementation capabilities and network interconnection technology support.
System Integration: Celestica is responsible for final board manufacturing and rack-level physical integration.
Devouring the entire industrial chain: OpenAI’s ambition for a full-stack empire
Train the model yourself, design the chip yourself, optimize inference yourself, and control deployment yourself.
Clearly, OpenAI's goal is a vast, end-to-end AI empire.
But OpenAI’s ambitions are even more ambitious than Apple’s and Google’s—they have an unprecedented super flywheel: using AI to accelerate the building of AI infrastructure, then using that stronger infrastructure to run even more powerful AI.
According to OpenAI's announced roadmap, the first gigawatt-scale super data centers will begin deployment in late 2026 in collaboration with core partners such as Microsoft.
The entire electricity consumption of a medium-sized city will be used to power the inference racks for Jalapeño and the next-generation pepper chips.
Get ready—soon, we’ll see GPT-5.6 Sol racing across the Cerebras wafer at 750 tokens/s, shattering the physical limits of parameters and inference speed.
Reference: https://x.com/bleysg/status/2073937651150029084
This article is from the WeChat public account "New Intelligence Yuan," authored by ASI Revelation; edited by Aeneas.