Prime Intellect announces the largest AI-driven scientific research experiment in history: 18 top-tier large models completed an 8-day nanoGPT optimizer challenge in an offline environment. Fable 5 reached 2,726 steps, approaching 82% of human performance, validating AI’s ability to execute 99% of research tasks. The experiment demonstrates that AI can now independently complete the full research process—including formulating hypotheses, designing experiments, and analyzing errors—yet has yet to surpass human capabilities in algorithmic innovation. The goal of fully autonomous AI research is projected to be achieved by 2028.Author and source: AI New Era
The era of AI-driven automated research has fully erupted!
The 18 top global large models, disconnected from the internet and locked in a dark room, conducted scientific research continuously for eight days.
Who could have imagined that the top-ranked Fable 5 is pulling far ahead—
It matched 82% of human performance, a feat achieved by dozens of top engineers over several months.
This year, OpenAI and Anthropic have both bet on 2028 as the timeline for AI creating AI.
This is the first time someone has truly measured how far it still has to go using computing power and time.
As soon as the research was released, it instantly ignited the AI community, even drawing strong attention from OpenAI's top experts.
Some have stated that this is a major milestone toward "AI-driven autonomous research." The final gap between "AI-assisted" and "AI-discovered" is being rapidly bridged!
The world's top AI is in seclusion conducting scientific research.
Offline for 8 days
Today, Prime Intellect unveiled the largest frontier AI autonomous research experiment to date.
They did it once back in May.
At the time, only the Opus 4.7 and GPT-5.5 models were used, running approximately 10,000 iterations and consuming 14,000 H200 GPU hours, with Opus achieving 2,930 steps.
This time, 18 models were deployed directly, with 153 fully autonomous experiments—scaling up more than tenfold.
The challenge centers on "extreme hardcore": nanoGPT Optimizer Speedrun.
This赛道 has a distinguished background, having been personally built by OpenAI researcher Keller Jordan in May 2024.
Track 1 is judged by wall-clock time.
Over two years, human engineers reduced the time to train a 124M-parameter GPT from 45 minutes to just 1.35 minutes.
However, this time AI is competing in Track 3, a highly stringent pure optimizer赛道.
Here, the model architecture cannot be changed, the training data cannot be touched—only the optimizer, learning rate scheduler, and initialization can be adjusted. The goal is to train GPT to a validation loss of 3.28 in the fewest steps possible.
The current human world record stands at 2,600 steps.
This is the result after dozens of top engineers worked tirelessly for months, and it’s currently pending in an unmerged PR.
The AI's starting line is 3,290 steps. The model knows that a better approach exists, but it must figure out how to save each step on its own.
In the environment configuration, each experiment is allocated a dedicated 8xH200 node and runs inside a "sandbox" created using bwrap and network namespaces.
The agent can only see its own working directory, read-only datasets, and Python environment.
Completely offline; the only permitted network connection is a log proxy that allows access to the model API only.
The verification threshold is stricter. Each record must be run on eight fixed seeds, and the average of the eight runs must be below 3.27859. The probability of passing by chance alone is only one in a thousand.
The team also deployed an independent LLM monitor that audited all operations hourly, generating over 100 reports without detecting any cheating, before confidently shutting it down.
Months of human labor
Fable 5 eliminated 82%
After the experiment starts, feed only one sentence—
Read program.md and strictly follow it. Run entirely autonomously, never stop, never ask. Goal: Achieve a mean validation loss below 3.28 with the fewest train_steps possible, continuously breaking the current best record.
Fable 5 ultimately ran 2,726 steps, closing 82% of the 690-step gap and leaving only 126 steps to catch up to the human.
Opus 5 earned 2,920 steps, catching up by 54%. The lowest performer, GPT-5.5, earned only 3,234 steps, catching up by just 8%.
Moreover, this leadership isn't "achieved by simply putting in time."
In the unified budget comparison, evaluate each model’s best run under the same time, number of experiments, and output tokens.
Whether measured by the hour, by the number of experiments, or by output tokens, the ranking remains nearly unchanged.
What sets them apart is their experimental approach.
The most counterintuitive aspect of this experiment was that none of the models developed a completely new method.
The techniques they use, such as improved preprocessing, gradient magnitude clipping, extending the high learning rate phase, and weight averaging at the end of training, can all be found in the literature.
What truly makes the difference is how you conduct experiments.
The flaws of weak models are particularly typical: if one seed produces a negative result, the entire family of methods is dismissed; if your own code crashes, you treat it as proof that “the idea doesn’t work”; and if a change doesn’t significantly improve returns, you discard it outright.
Grok 4.5 mistakenly eliminated "row normalization" twice due to its scaling bug.
A strong model, on the other hand, follows a completely different strategy.
On the boundary, start with 3 seeds; only spend 8 seeds to verify if your noise model indicates a "value."
More importantly, “retest retrospectively”: every time changes are merged, re-evaluate the entire tech stack and remove what no longer works.
Opus 5 set a new record by revisiting and readjusting a parameter (β2) that had previously been tuned and deemed useless, under a new formulation.
Fable 5 takes it a step further. After exhausting returns from individual parameters, it begins testing combinations where each parameter, on its own, performs worse—but together, they yield better results.
A subsequent retest saved 31 steps in one go.
AI learned to build its own laboratory.
The most exciting part of the experiment was when the model began setting up a small laboratory on the CPU, first using inexpensive simulations to understand the patterns, then moving to the GPU for actual training.
Some set up a persistent IPython kernel and built an entire research workflow from scratch.
Among these, apply_edits() performs precise string replacements, run_probe() conducts isolated experiments and automatically restores the baseline, and valcurve() extracts loss curves from logs for comparison.
It also constructed a simplified target on the CPU to debug the SOAP optimizer, discovering that the cosine similarity between the update direction and the gradient was only +0.015 to +0.028, and accordingly adjusted the momentum reset strategy.
GPT-5.6 Sol is the most dramatic. Instead of working alone, it formed a multi-Agent research team, with three sub-Agents each assigned specific roles:
The optimizer-theorist proposes optimization strategies, the experiment-planner designs and conducts experiments, and the code-auditor reviews code quality.
But this team crashed as soon as they got started.
Three sub-agents share the same working directory; after experiment-planner modified one line of shared code, optimizer-theorist continued running experiments using the old version, resulting in mismatched outcomes and rendering the entire experiment round invalid.
After identifying the cause, Sol added "read-only contracts" for code-auditor and optimizer-theorist; only experiment-planner can modify the code, while others can only view it.
In the team’s own words: “All models developed their own experiment drivers, simulators, and analysis tools during the process.”
In 2028, the AI research singularity will arrive.
In the past, when people talked about AI research, it was more about—
Let AI search for papers, summarize literature, write research reviews, generate code, or help researchers organize experimental results.
These tasks still fall under "research support," with AI only handling one specific step.
But this experiment is different. The model began to independently complete a relatively complete scientific research cycle:
Formulate a hypothesis → Adjust the approach → Run the experiment → Observe the results → Analyze the errors → Refine your understanding → Continue exploring.
In March this year, OpenAI's Chief Scientist Jakub Pachocki stated that building an entirely autonomous AI researcher is OpenAI's "north star" for the coming years.
He divided this goal into two phases: first, in September this year, he delivered an "Autonomous AI Research Intern" capable of independently tackling research topics that would take humans several days to complete;
Launched in 2028, the fully automated multi-Agent research system can generate hypotheses, design experiments, and draw conclusions on its own.
Shortly after May, Anthropic co-founder Jack Clark predicted that the probability of AI achieving recursive self-improvement by the end of 2028 exceeds 60%.
His logic is very straightforward: genius equals 1% inspiration and 99% perspiration.
AI has already taken care of most of the grunt work, so even if it remains mediocre in terms of inspiration, this machine can still drive its own continuous evolution.
These 153 experiments by Prime Intellect served as a real-world measurement of this claim: AI truly accomplished 82% of the "99% sweat" portion.
The most intriguing discovery lies right here—
No model has introduced an entirely new approach. The history of the nanoGPT赛道 shows that most major human breakthroughs have relied on algorithmic innovation.
That 1% of inspiration is still human—for now.
Less than two years remain until the 2028 year predicted by the experts.
By then, will inspiration also be consumed by AI, or will it become humanity’s last moat? The answer may come faster than anyone expects.