Karpathy Warns AI Developers: Focusing on Agents Before Models Is a Mistake

Karpathy’s internal explosion: Forcing agents to work is AI’s biggest mistake! The cutting edge isn’t at OpenAI—it’s in your hands.

A single sentence left the entire Agent community thoroughly chilled.

Andrej Karpathy—currently a core researcher at Anthropic’s pretraining team—recently dropped a bombshell during a live session for agent developers that left the audience silent:

The biggest mistake in today's AI field is that people rush to make agents work without first fully understanding the underlying large models.

This clip was cut and posted on X, and within days it went viral.

Because it targets precisely the hottest, most crowded赛道 that everyone is rushing into right now.

And the person saying this isn’t a novice dampening spirits—they’re someone who’s been burned, reflecting on their own hard-earned lessons.

Go back to the year 2016.

At that time, Karpathy was working on a project at OpenAI called World of Bits, with a goal that sounded particularly “2026”: training an agent to use a keyboard and mouse to operate a computer, book flights, order food, and get tasks done for you.

Familiar? This is almost the exact image on the first slide of every Agent startup’s pitch deck today.

So what happened? It didn't work out.

Karpathy put it plainly: At the time, he and Tianlin Shi, Jim Fan, and a few others were frantically clicking on a handful of basic web pages, trying to book a flight and order food—and somehow, they ended up publishing a paper at ICML 2017.

The paper is titled "World of Bits: An Open-Domain Platform for Web-Based Agents"—a grand vision of the "World of Bits" ultimately trapped within a few rudimentary web pages.

The technology isn't ready. The only hammer I have is reinforcement learning, and no matter how hard I swing, I can't make it work.

Looking back, the truly correct approach at that time was to completely forget about agents and shift focus to language models.

Five years later, the toolbox has been completely overhauled—you now build Agents, and almost no one uses reinforcement learning anymore. Back then, this was unthinkable.

Interestingly, Jim Fan, who co-authored the paper with him back then, is now a senior research scientist at NVIDIA and has developed a series of groundbreaking projects such as Voyager and MineDojo, earning a distinguished paper award at NeurIPS.

A young intern from a 2016 "failed project" became a top player in the AI Agent field a decade later.

But the path taken is not the one from 2016.

The demo is easy, but the product takes ten years.

Following this lesson, Karpathy offered three pieces of advice, each going against the current trend.

First, stop pushing your agent to do everything—get the underlying model right first.

When he joined Anthropic’s pre-training team in May this year, his first post on X was: I believe the work at the frontier of LLMs will be especially critical over the coming years.

Someone who "invented" vibe coding and had it named Word of the Year by Collins Dictionary has now chosen to return to the most fundamental pre-training research—this itself is a "behavioral vote" against the Agent hype.

Step two: The demo is easy, but turning it into a product will take ten years.

He cited two well-known examples: autonomous driving—anyone can create a demo of a car driving around a block, but turning it into a real product took a full decade; he personally experienced this marathon at Tesla.

The same goes for VR—stunning demos are everywhere, but turning them into real products still takes at least a decade.

Agent, this type here.

Very easy to imagine and create a demo, but extremely difficult to turn into a real product.

If you're truly entering this industry, be prepared to work for ten years—not just finish a cool demo and think you've made it.

Step three: The agent is not the product; the foundational capabilities are. Strengthen the foundation, and agents will naturally emerge.

These three statements completely reject the current trend of simply wrapping a shell around something, stacking up agents, and rushing to launch.

Karpathy's point is clear: the weaker the foundation, the faster the building collapses.

Autonomous driving has already been validated for everyone over the past decade; there’s no reason why agents should skip this lesson.

Learn from the brain

After sharing the lesson, Karpathy shifted gears and dove into neuroscience for inspiration.

He posed a series of questions on stage: What in an agent corresponds to the hippocampus, responsible for memory, indexing, and retrieval?

What corresponds to the basal ganglia, which controls behavioral selection and action execution? What corresponds to the thalamus, the place where “multiple thoughts compete for the microphone,” like the seat of consciousness?

A top AI researcher says: "In creating digital life, what we most lack right now is not more flashy features, but reverence for the fundamental question: 'What is intelligence?'"

He even brought along a neuroscience book by David Eagleman, titled "Brain and Behavior: A Cognitive Neuroscience Perspective," and recommended it to everyone present.

In his view, building agents today is as worthwhile as the early days of deep learning—back then, we drew inspiration for artificial neural networks from the structure of individual neurons; today, we can once again draw inspiration from the brain.

What really stole the show was that final line.

If the previous remarks were a cold shower, Karpathy’s conclusion ignited a fire under the audience.

He said to the room full of independent developers and entrepreneurs:

You are the ones truly at the forefront of Agent capabilities—not OpenAI, not DeepMind, but you.

This isn't just polite small talk. He gave a particularly painful explanation:

Large companies like OpenAI are unmatched in training large-scale Transformer language models—when a new Transformer training paper is released, the typical reaction inside their Slack is: “Oh, someone tried that two and a half years ago; we know exactly why it didn’t work.”

But as soon as a new Agent paper emerges, everyone's reaction is: "Oh, that's cool, that's novel."

Why? Because no major company has accumulated five years of experience in Agent technology.

Big companies don’t stand on the edge of capability—you, the entrepreneurs and hackers—do.

The concept is actually not hard to understand.

Big companies have spent years running down the path of language models, having already encountered every pitfall and mapped out every detour; but agents are a newly cultivated frontier, where no one has a five-year head start, and everyone is essentially on the same starting line.

At this moment, independent developers who are flexible, willing to experiment, and able to pivot quickly have a better chance of breakthroughs than large corporations that are hard to turn around.

The cold water Karpathy wants to pour is not "Don't build agents," but "Don't skip the fundamentals to build agents."

He himself is the best illustration—this inventor of vibe coding, who pushed agents to their limits, chose in 2026 the most important career move: returning to pre-training, back to the lab at the very foundation of large models.

The fire he lights isn’t meant to cause anxiety, but to tell everyone fighting on the front lines: You’re not falling behind—you’re right at the forefront.

Hype always fades, and demos eventually lose their luster.

Only those who deeply understand the underlying model and are willing to dedicate themselves to a single pursuit for a decade deserve to stand on the shore ten years from now.

Reference: https://x.com/0xCodila/status/2073544407643496771

This article is from the WeChat public account "New Intelligence Yuan," authored by ASI Revelation, edited by Solomon.