Claude Fable 5 Dominates AI Coding Benchmark with a 64% Success Rate

This time, Claude has truly shattered the AI programming rankings!

Just refreshed on the MirrorCode leaderboard, Claude Fable 5 has reclaimed the top spot with an absolute success rate of 64%.

GPT-5.6 Sol, which follows closely behind, has only one-third of its score!

Ranked fourth, GPT-5.5 fared even worse, scoring only 10% and being thoroughly outperformed by its own predecessor, GPT-5.4.

More surprisingly, when using high-resource languages like Go, Fable 5 achieved a solution rate of 64%; even when switched to the obscure language Ada, the performance remained as high as 61%.

In the open-source world, the Python corpus is approximately 230 times larger than that of Ada.

But at Fable 5, the performance only dropped by 3 percentage points.

That's interesting.

If the model primarily relied on memorizing grammar and common writing patterns in popular languages, its performance should have decreased after switching to Ada.

But the current outcome points to another possibility—

The most powerful models have already broken free from reliance on specific training data and are beginning to learn how to build complete software projects from scratch.

Specifically, the complete MirrorCode includes 25 target programs covering areas such as Unix tools, interpreters, data querying, bioinformatics, cryptography, and compression tools.

Here, the model is placed in an isolated environment with no internet access, unable to view the original project's source code or download third-party dependencies. It can only access high-level documentation, some visible tests, and a reusable original program.

Next, it continuously feeds data into the original program, observes the outputs to infer the internal logic, and then Click here Write a new program with consistent behavior.

Select 15 Medium and Large targets from the latest ranking. Each target is implemented in two programming languages, with each language executed three times.

Additionally, both visible and hidden tests must achieve a 100% completion rate to pass; 99.9% is not acceptable.

Missing even one edge case still counts as a failed run.

To compel the model to fill in the final few gaps, MirrorCode increased the single-budget limit to 10 billion tokens and allowed continuous operation for up to seven days.

The most expensive task in the paper was even more extreme: the model ran continuously for 19 days, costing $2,600 in a single run.

Over these 19 days, the model repeatedly ran the original program, compared results, added missing features, and reran tests. When errors occurred, it investigated the cause; when outputs didn’t match, it adjusted its assumptions; and once a section passed, it moved on to address the next gap.

The entire process is more like a multi-day debugging session than a single generation.

The gotree example is the most intuitive.

This bioinformatics tool originally had approximately 16,000 lines of Go code and over 40 commands.

Claude Opus 4.7 spent 14 hours and $251, passing 2,000 out of 2,001 tests with a completion rate of 99.95%.

Although it was halted by MirrorCode’s 100% pass line due to overlooking a niche edge case involving date processing, it reduced what was originally a weekly workload to just a few dozen hours—

Epoch estimates that without AI, human engineers would take at least 2 to 17 weeks to complete the same task.

The MirrorCode paper previously used the publicly available training mixture from StarCoder as a reference.

Python accounts for approximately 8%, while Ada accounts for only 0.034%, making the former about 230 times larger than the latter.

Of course, we cannot know exactly how much Ada code the closed-source model has seen. But looking at the open ecosystem as a reference, the scarcity of Ada is already clear enough.

This language primarily appears in aerospace, defense, and other safety-critical systems. It far lags behind Python, JavaScript, or Go in terms of community size, number of tutorials, and open-source projects.

And Fable 5 is clearly not translating Go code line by line into Ada.

It’s more about first understanding how the original program works, then rewriting the same behavior in a different language.

In comparison, other models are not as stable.

GPT-5.6 dropped from 24% to 19%, GPT-5.4 fell from 21% to 12%, and GPT-5.5 plummeted from 17% to 5%. Switching languages immediately amplified the gap.

Today, Cursor has enabled hundreds of agents to collaborate for nearly a week, writing over one million lines of browser code from scratch, spread across 1,000 files.

Anthropic ran 16 Claude Agents in parallel for nearly 2,000 conversations, ultimately building a 100,000-line C compiler capable of compiling the Linux 6.9 kernel.

Among the individual user samples of Codex disclosed by OpenAI as of May, 70.2% submitted at least one task estimated to require more than one hour of human effort; 25.6% submitted tasks estimated to require more than eight hours.

The units people entrust to AI are shifting from a piece of code or a single bug to an afternoon, a week, or even an entire project.

64% of MirrorCode is a quantification of this type of "project-level delegation."

It indicates that, with sufficiently clear objectives and automatically verifiable outcomes, state-of-the-art models can now independently complete portions of medium to large-scale software.

For everyone entrusting their work to AI, change is already upon us—

Previously, you had to closely monitor and write every line of code; now, you're more likely to simply check at key milestones to ensure it hasn't gone off track.

The code will become cheaper.

Those who can clearly explain problems and verify results will become increasingly valuable.

Reference: https://epoch.ai/MirrorCode

This article is from the WeChat public account "New Intelligence Yuan," authored by ASI Revelation; edited by Moses.