CUDA has been cracked again...
This time, it’s AI that NVIDIA itself nurtured that’s taking action.
A small company, founded just one year ago, used an AI programming agent to build a "CUDA-like software" in only 10 hours.
Stop, stop, stop.
You're saying that this software empire built by NVIDIA over nearly 20 years has just been copied?
And more than that. DeepSeek Also trying to write less low-level CUDA.
They have open-sourced a GPU operator library called TileKernels, written in TileLang, which eliminates the need for extensive manual writing of low-level CUDA code.
But we’re not hearing about a similar “CUDA crisis” for the first time.
Every so often, CUDA gets dragged out and criticized, with claims that it’s been replaced, breached, or that its moat has been overturned…
Is that really true?
10-hour "replication" of CUDA
When people think of NVIDIA, their first reaction is often GPUs.
With successive generations of more powerful chips—the H100, Blackwell, and Rubin—NVIDIA has captured the largest share of the benefits from this AI wave.
But NVIDIA's true, hard-to-challenge advantage lies not in hardware, but in software.
Chips can be purchased, specifications can be caught up on, and manufacturing processes will continue to evolve. What truly makes it difficult for customers to switch away from NVIDIA after using its products is the comprehensive software ecosystem built around its GPUs.
And CUDA is the core of this software empire.
CUDA stands for Compute Unified Device Architecture and was developed over nearly 20 years by a team led by NVIDIA executive Ian Buck.
It is often referred to as a programming language, but more accurately, CUDA is an entire software platform built around NVIDIA GPUs.
If broken down simply into three layers, the bottom layer is the core layer, consisting of the kernel, compiler, and runtime that directly command the chip.
The middle layer consists of highly optimized computational libraries such as cuBLAS and cuDNN, along with debugging, validation, and performance analysis tools.
At the top are development frameworks such as PyTorch and TensorFlow, vast amounts of enterprise-coded assets and workflows, and a developer ecosystem built around CUDA.
△
This might sound a bit abstract, but you can think of NVIDIA GPUs as a factory.
At the lowest level, CUDA tells the machine how to perform each step; at the intermediate level, it provides ready-made production tools; and at the highest level, it offers a fully established production system that has been operating for years.
So, for customers, what they’re buying isn’t just a single NVIDIA GPU—it’s a turnkey factory.
Now, a guy named Jeremy Nixon has arrived with an AI Agent.
Nixon is the founder of the AI software startup Infinity and previously worked as a researcher at Google Brain.
His team developed an AI research agent called Ignition, specifically designed to write low-level software for different AI chips.
It can generate GPU kernels, run tests, identify bugs, measure performance, and automatically rewrite code based on the results.
Human engineers are responsible for setting the high-level direction, while the tedious and mentally demanding tasks are handed over to agents for repeated cycles.
That's it—Infinity built a CUDA-like software stack for AI chip startup d-Matrix using Ignition.
It only took 10 hours.
Huh??
Can the low-level code that once required chip software engineers to repeatedly debug really be entrusted to AI now?
It really can't be said to be entirely hype.
AI agents are indeed well-suited for programming tasks with clear feedback.
Whether the code compiles, whether the results are correct, and whether performance has improved can all be determined through actual execution.
Thus, the agent can form a complete闭环:
Write code → Compile → Run → Test for correctness and performance → Iterate based on feedback
However, Infinity primarily targets the first layer of CUDA: generating and optimizing inference kernels for different chips, and building underlying software capabilities around these kernels.
It is developing a universal inference library compatible with multiple chips, but this does not mean it has replicated CUDA’s entire ecosystem, built up over nearly 20 years, in just 10 hours.
But…
But!
You don’t actually need to copy CUDA to raise $15 million in funding.
The company's current valuation has reached $100 million.
I don’t know if there’s a threat to NVIDIA, but capital is definitely buying in. (doge)
Inference side, the second front is launched!
If AI agents are the weapons for breaching the city walls, then the reasoning market is the gap in those walls.
The computation of large models is primarily divided into two stages:
Training involves building a model and requires thousands of GPUs working together for months; inference uses the trained model to answer questions and can run on a small cluster or even a single GPU.
On the training side, CUDA remains nearly unsolvable.
Large-scale training is highly sensitive to performance, stability, and cluster coordination. Any efficiency loss in communication between chips, memory scheduling, or recovery after program failures, when amplified across thousands or even tens of thousands of chips, becomes real time and cost overhead.
Therefore, businesses tend to be extremely cautious when choosing a training platform.
Even if other chips have impressive specifications on paper, if their software isn't mature or their clusters aren't stable, the saved hardware costs will likely be doubled by development and trial-and-error expenses.
But on the inference side, the rules have changed.
Marshall Choy, Chief Business Officer of South Korean AI chip company Rebellions, believes:
On the inference side, CUDA is no longer the deciding factor. This will be a competition among open-source software.
Why are competitors all focusing on inference?
Training prioritizes "maximum performance," while inference focuses on "cost per response."
Training requires thousands of GPUs working together, while inference can be split across small clusters or even a single GPU; you dare not switch chips for training, but you can for inference.
As long as it runs smoothly and is affordable, customers are willing to try it.
More importantly, inference software can run across different chips. If the inference framework supports multiple types of chips, users can switch between different hardware without rewriting their code—
The greatest lock-in effect of CUDA has now been eliminated.
This creates an opportunity for specialized inference chips.
Not all inference tasks require the full capabilities of CUDA, from training to cluster coordination. Chips redesigned for specific models and scenarios have the opportunity to capture market share from NVIDIA if they can run faster, cheaper, or more energy-efficiently.
For example, d-Matrix is a chip company that specializes in inference.
The company was founded in 2019 and specializes in generative AI inference chips.
Co-founder and CEO Sid Sheth recalled that they early on recognized that the opportunities brought by AI inference would eventually surpass those of training.
Infinity used an AI agent to build a CUDA-like software in 10 hours, specifically designed to serve d-Matrix's inference chips.
Thus, the full story of the "10-hour incident" comes together:
A new chip aims to enter the inference market but lacks mature software; AI agents rapidly generate and optimize underlying kernels, reducing what could have taken months or even years of adaptation to just hours or days.
Sid also openly stated:
Although NVIDIA maintains dominance in training, its moat has weakened in inference.
d-Matrix is not the only one targeting this gap.
Rebellions and d-Matrix, focused on inference; Cerebras, betting on wafer-scale chips; Amazon’s Inferentia, Google’s TPU, AMD’s MI300X…
Various chip manufacturers and cloud computing giants have already positioned themselves in the inference market, ready to take a slice of the pie from NVIDIA.
For these companies, there is no need to immediately replace NVIDIA.
They only need to demonstrate that, for certain reasoning tasks, it's possible to achieve faster or cheaper performance without relying on NVIDIA's full hardware stack and CUDA ecosystem.
That's enough.
Has NVIDIA's moat been disrupted?
The answer might be... still far off.
As mentioned earlier, the 10-hour rewrite was for "adapter code for a single chip," but the CUDA ecosystem also has two decades of accumulated libraries, debugging tools, community, and millions of developers.
This is not something that can be easily overturned.
More importantly, just because code can be generated doesn't mean it can be used.
Bing Xu, founder of INT21, previously founded HippoML, an AI chip software company acquired by NVIDIA, and he left NVIDIA just this past April.
His assessment is: Agents can generate large amounts of code in a short time, but verification remains the biggest bottleneck.
AI can generate ten thousand lines of code quickly, but proving that those ten thousand lines compute correctly and run stably under all edge cases is extremely slow.
CUDA's deepest asset is precisely its verification toolchain—he believes that in the Agent era, the verification ecosystem will become CUDA's next moat.
Modular co-founder and CEO Chris Lattner also poured cold water on the idea.
He believes the improvements brought by coding agents are incremental, and "the hype has been greatly exaggerated."
There are three reasons: First, writing code is only a small part of software engineering; production-grade optimization determines chip performance and directly impacts the cost of running AI.
Second, chip software is a niche, elite field with far less open-source code compared to application development, so there is inherently less training data available for AI to learn from in this area;
Third, the cost of migrating millions of lines of existing code still remains; this is not a technical issue but an organizational decision.
One easily overlooked point is that NVIDIA itself is also using AI.
Ankit Patel, Vice President of NVIDIA's Developer Ecosystem, said:
We are also using AI agents to accelerate CUDA development and conduct validation at a larger scale.
Using your own spear to attack the other's shield also diminishes the attacker's relative advantage.
Bing Xu concludes: The outcome of this battle depends on whether competitors can catch up to NVIDIA faster than NVIDIA can iterate on itself.
But clearly, the world's largest chip manufacturer "has not been sleeping or standing still."
Overall, NVIDIA's moat remains secure in the short term, but changes will first occur quietly on the inference side.
The real long-term question is: the moat is shifting from "code存量" to "verified and optimized ecosystems."
Whoever claims the new position first will be the next winner.
Reference link: [1] https://www.businessinsider.com/nvidia-cuda-new-threats-ai-coding-agents-2026-8
This article is from the WeChat official account "Quantum Bit," authored by Ting Yu.