Former Core Researcher at OpenAI: Human AI Research May Be Replaced Within Two Years

Hey, have you heard?

Human researchers have at most two years left.

Two years from now, humans conducting AI research will be like humans playing chess today.

You can still place orders, but no one cares how you place them.

This is the latest controversial statement from Jerry Tworek, former lead of OpenAI's reasoning model.

He joined OpenAI in 2019 and stayed for seven years. When reinforcement learning was stubbornly resistant to scaling, he persisted relentlessly and pushed it forward; he personally led the teams that developed the groundbreaking models O1 and O3.

“We want to build AGI.” That was Ilya’s entire roadmap when he joined OpenAI in 2019, presented at the all-hands meeting. Seven years later, he left to start his own venture.

Even more chilling is that he’s not the only one saying this. The most popular inside joke among AI researchers right now is:

We only have a few more days of work left—while we still can, let’s finish up quickly, then all retire and take a well-deserved break.

Everyone keeps retelling it as a dark joke, but each person telling it knows full well the truth.

This joke is probably reality.

And this was just the appetizer. Throughout the entire interview, he delivered one tough statement after another:

• The agent does exhibit creativity, but it’s an extremely poor, overly abundant kind—ideas are wildly varied but generally terrible.
• Globally, perhaps only thirty to fifty people have truly taken a cutting-edge model from training to deployment; everyone else is supporting them.
• Assuming that the Transformer is the optimal solution is almost certainly unrealistic.
• During the seven years at OpenAI, there were only three to four serious attempts to replace the underlying architecture.
• The assumption that the world depends on us working to keep moving is fundamentally invalid.

Here is the revised version. Enjoy.

This figure over two years has nothing to do with benchmark scores or computing power.

He counts how many tasks in this research are still being done by humans.

And this task has long been split into two parts.

One part is coming up with ideas and figuring out where to go. The other part is doing the work—turning those ideas into runnable code and retrieving the data.

And that part of the work has already been largely taken over by the Agent.

In April this year, Tworek founded a company called Core Automation, with the slogan of building the world's most automated AI lab.

On their platform, the complete cycle of an experiment was reduced from one month to just one day—efficiency improved by a full 30 times!

However, Agent is currently unable to handle this part of the suggestion. The reason is—

They definitely belong to a highly creative species.

But they are squandering creativity in an extremely poor, overly abundant manner. The ideas are indeed diverse, but they are普遍 terrible.

Listening to this, the part about coming up with ideas still seems stable.

Actually, there aren’t many people left who can truly offer good advice.

During the interview, the host revealed that an internal researcher at OpenAI had confided in him—

Globally, there may only be thirty to fifty people who truly understand how to train and deploy a cutting-edge model from end to end.

Everyone else is supporting these dozen or so minds, including the vast majority of full-time employees at the most impressive company.

Tworek did not object.

He also believes that any top team operates this way: a few individuals set the course, pulling behind them a whole roaring machine of execution.

To understand how in-demand these dozens of people are, just look at his own recruitment list.

Rohan Anil, co-founder of Core Automation, previously from Anthropic and earlier at Google DeepMind.

Anmol Gulati, who worked on Gemini at DeepMind, was also poached. Even Julia Villagra, OpenAI’s former head of people, followed them over.

All labs are fishing from the same pool, and there are only dozens of fish in that pool.

So this number of two years has nothing to do with all of humanity.

It’s about these dozens of people—how much longer they can hold on.

Since there are only dozens of people left, what is the last barrier standing in AI's way?

Tworek's answer was just one word: Transformer.

In his view, this solution is almost certainly not optimal.

First, the model cannot continue learning after deployment. No matter how much you chat with it, it won't get stronger—it will remain the same in the next conversation. The context window also can't handle it; he says that after using Codex for about twenty minutes, he has to compress it.

Second, if you try to compensate through continuous fine-tuning, you’ll find that it’s extremely inefficient and leads to catastrophic forgetting—learning new information causes you to forget the old.

So over the past few years, most of the industry’s efforts around Transformer have been focused on making it cheaper, with very few truly working to make it stronger.

But what Tworek truly wants isn't actually in the architecture itself.

He wants a model that can continue learning during test time—a model that grows continuously from user interactions and user data. Changing the architecture is merely a means to achieve that goal.

Everyone in the industry understands these principles. But after all these years, why is Transformer still firmly in place?

The reason is so simple it’s almost absurd—there was hardly any attempt at all.

Over seven years at OpenAI, there were only three to four serious attempts to replace the underlying architecture.

The process is as follows.

The researcher first conducts a small-scale validation experiment and runs it for at least three months. Only when the results look promising do they proceed to scale up.

And by “leverage,” what’s meant is that you must use your persuasive skills to convince about ten people with decisive authority, and then those ten individuals invest three to six months’ worth of resources into your bottomless pit.

Either it will be crushed by the momentum of the snowball that Transformers have long since rolled, or it will be partially absorbed, becoming a screw in its structure.

Seven years, three or four times. This is the entire capacity for architectural innovation at the world’s most powerful AI lab.

Jare, take these hashrate units and mine.

He himself had previously hit the exact same dead end at OpenAI—and it was one sentence that broke it open.

At the time, that batch of half-dead experiments managed to show a glimmer of signs of life. It wasn’t great, but at least there was a beginning.

At this critical moment, Jakub Pachocki, who later became the Chief Scientist at OpenAI, found him.

Jare, take all these GPUs—you see if you can push your results even bigger and harder.

"You now have these GPUs." He repeated this sentence when o1 didn't yet have a name.

This pulled him out of a paradox—

You must deliver results first before you’re eligible to request computing power.

But you must first have that computing power to produce the result.

Tworek said that most cutting-edge directions are trapped in this paradox. The covert battles for computing power in laboratories are ultimately about finding a way out.

And sometimes, the export is just a decision made by the leadership. Click Confidence.

As long as someone above says, “I want you to have a decent hash rate allocation to go all in,” that’s all it takes.

Once the gate opens, it becomes impossible to hold back what follows.

Reinforcement learning has increased by several orders of magnitude starting from this point.

Then, o1 emerged like a butterfly breaking free from its cocoon, breathing new life into the path of reasoning models—once declared dead by countless others.

One hundred thousand dollars equals one expert.

This time, he didn't have to wait for anyone to speak up.

The most expensive part of the entire chain is translating abstract ideas into executable code—and this is precisely what today’s agents do best.

For example, Core Automation once applied this approach to tackle GPU cores.

Specifically, they handed a QR decomposition kernel to a programming agent, ran it for four weeks, spent approximately $100,000 in invocation costs, and ultimately increased the kernel’s speed by a factor of sixty.

This work falls under low-level performance engineering, focusing on optimizing how a matrix operation runs faster on a GPU—typically requiring only a handful of experts to manually fine-tune it line by line.

And there are only a few such experts worldwide.

Now, one hundred thousand dollars can stand in for it. It requires no persuasion and no three to six months.

The barrier that had held for seven years was finally broken through.

Even the most stubborn architecture has begun to loosen, so the steering wheels in the hands of those dozens of people won’t remain under their control for long.

At the end of the interview, the host directly asked him: when that day truly arrives, what will people still have left to do?

For this, Tworek described two scenarios.

The first one is ancient Greece.

Everyone meets up in the square, leisurely discusses philosophy all day, then heads off to exercise, snacking on olives and sipping wine.

He smiled as soon as he finished describing it, admitting that it was probably just projecting his own desires.

“Let’s meet up in the square and chat.” He quoted his own words describing daily life in the post-AGI era, then laughed at himself.

The second one is high school, or college.

He believes people should maintain a pursuit of "excellence" itself—continuing to greedily learn new things and pushing both body and mind to their limits.

It’s a bit like professional sports—there’s no economic reason forcing you to sweat and bleed, but human pride drives us to reach for something called “greatness.”

We need to find various ways to do this.

In the next world, there won't be any more situations where "if you don't do it, the world will collapse." The world will continue running on the infrastructure we've already built.

“We should keep learning forever.” He spoke of it as an obligation rather than a pastime.

Then he calmly revealed his cards.

We must work for the world to function. This assumption doesn't need to be true.

Many people derive their self-worth from their work. He acknowledged that this was the hardest hurdle to overcome, even for himself.

To be honest, after listening to the whole thing, what’s most unsettling isn’t the two years.

He knows full well that he’s stepping on the gas for this countdown.

The thing he’s building is defined by its ability to learn and improve on its own—no longer requiring people to constantly feed it data. Once it’s built, those几十个人’s roles will disappear even faster.

And he has one even more severe statement: he will be the first to be judged.

If you’re not the lab with the most formidable computing power and the largest scale, you’ll be left far behind.

At the time this was said, Core Automation had been in operation for only four months and had zero revenue on its books.

He said that since starting his business, almost every week someone would come to him and say, “Jare, it’s too late—the ladder to heaven has already been pulled up.”

His response was only one of his usual catchphrases.

The thing our company loves to say is that everything is a skill issue.

In the end, don't blame the market conditions—blame yourself for lacking the skills.

You know what? This statement actually fits quite well in this interview.

Look back again at that popular meme at the beginning.

We only have a few more days left to work—now’s the time to get it done while you still can.

We can’t tell whether the tone was one of excitement or sorrow.

Because, for the person saying this, those two emotions are essentially the same.

Tworek said that the pace of this line is extremely draining; after all these years, it’s truly, truly exhausting.

But if we truly believe, this is the most important period of our entire careers.

That would probably be well worth it.

Reference materials:

https://x.com/MTSlive/status/2092387349623935322

This article is from the WeChat public account "New Intelligence Yuan," authored by ASI Revelation, edited by Moses.