Anthropic Acknowledges Responsibility as Claude Code and Opus 5 Encounter Performance Issues

icon MarsBit
Share
AI summary iconSummary
Anthropic faced backlash after a developer revealed that Claude Code 2.1.237 incorrectly mapped the 'high' reasoning level to the value of 10, which was previously used for 'low'. The error, part of an undisclosed A/B test, confused users. Anthropic engineer Thariq Shihipar confirmed the issue but stated it did not affect model performance. Meanwhile, Opus 5 has drawn criticism for inconsistent performance and a poor user experience. Engineers acknowledge the model is unstable and under active development. Crypto news continues to highlight growing scrutiny of AI tools in the cryptocurrency news space.

Claude became dumb overnight, sparking outrage across the internet!

Yesterday, developer argofowl spent the entire afternoon nearly turning Claude Code upside down.

Opus 5

He systematically investigated, first suspecting that the T3 code had crashed, then thinking there might be a bug in his own code.

In the end, he even began to wonder—had he broken his Mac himself?

When Argofowl finally opened the real API request logs, the truth was revealed—a number "10" was clearly listed.

Opus 5

However, on the Claude Code backend, he clearly selected 'high,' the highest reasoning level.

Who would have thought that the release notes for Claude Code contained not a single word?

Increase the inference high to 10

Claude Code being exposed as dumber

Argofowl found that starting from Claude Code 2.1.237, the model interpreted the "high" reasoning level as 10 out of 100.

And this number is precisely the value corresponding to the previous "low" tier.

Upon closer inspection, Anthropic has included Fable 5 sessions from Claude Code 2.1.236 and later versions in an experiment involving a "compressed effort scale."

However, the older versions and Opus 5 are unaffected.

Opus 5

This is most likely an A/B test, so not everyone will encounter it.

This is the real pain point for developers. You can tolerate a model being a bit stronger or weaker.

But you put me in the experimental group without telling me—so what exactly was I debugging this afternoon? Was I fixing my own code, or were you testing your A/B test?

Unexpectedly, after tech influencer Chubby shared it, the AI community went wild—

It seems Anthropic quietly made the model dumber without telling anyone.

Opus 5

For a while, self-test posts asking “Has Claude gotten dumber?” flooded X.

Someone posted a comparison of outputs from the same prompt across different versions; someone else retrieved their conversation logs from two weeks ago and performed a line-by-line diff.

Anthropic admits error, engineers step in

In response to the storm, Claude Code engineer Thariq Shihipar reacted quickly.

We sometimes test API service configurations in Claude Code first before deciding whether to roll them out fully.

The current experiment only changes how the effort values are mapped, so some people may see Claude say it is "10".

Opus 5

The key point is that this scale is not from 0 to 100; the number alone has no meaning—whatever effort you select is the effort you receive.

He emphasized that the team conducted thorough evaluations to confirm that this does not affect model performance.

Opus 5

Opus 5 is really dumbing down.

Just resolved the Fable issue, when Chubby bluntly said that Opus 5 now feels like a significant downgrade.

It always cuts corners and repeatedly makes basic mistakes. Once pointed out for not following instructions, it merely replies mechanically with that one phrase—

You're right; I was careless. Going over and over, endlessly.

In fact, several days ago, someone already noticed that Opus 5 was clearly exhibiting signs of "dumbing down."

In addition to the issues mentioned above, it also creates bugs, consumes large amounts of time fixing them, and repeatedly corrects itself within the same task...

Opus 5

Opus 5

After being pressed by netizens, engineer Thariq publicly acknowledged that Opus 5 is a "highly unstable" model, with erratic and inconsistent performance.

Our team is actively working to resolve this issue, and it is our highest priority.

Opus 5

Splitting always goes up

Feel like it's been going down all along

The Opus 5 controversy has exposed the most embarrassing flaw in the entire industry:

Splitting and user experience are systematically decoupling.

On one side is an outstanding, nearly flawless record: a composite score of 82.72, 79.2% on SWE-bench Pro, and 86.7% on Terminal-Bench.

On the other hand, users expressed completely opposite real feelings: "wordy," "lazy," and "argumentative."

The most absurd part is that both evaluations appeared simultaneously on Opus 5.

Moreover, "models becoming less intelligent" is not an issue unique to Anthropic.

Opus 5

Today, updates to large models have become the most opaque black box in the entire AI industry.

Traditional software uses semantic versioning, change logs, and rollback mechanisms. Developers can clearly know which version they are using and what changes have occurred.

Large models are different.

Under the same model name, the server may随时 conduct A/B tests, switch quantitative strategies, adjust model routing, or even change inference resources.

The only dashboard people have left is their own intuition.

Intuition, precisely, is the most easily dismissed and hardest to falsify thing.

The greatest value of this upheaval is that it has brought to light a long-standing underlying issue within the industry:

When models become infrastructure, stability is a covenant of trust. Benchmarks can be used for marketing, but stability can only be earned through consistent delivery.

Reference materials:

https://x.com/trq212/status/2091252347913773169?s=20

https://x.com/kimmonismus/status/2091178321669198014

This article is from the WeChat public account "New Intelligence Yuan," authored by ASI Revelation, edited by Peach.

Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.