Kimi K3 Closes in on Fable5 in the AA-Briefcase Benchmark, Task Costs Surge 10x

iconChainthink
Share
AI summary iconSummary
On-chain news: Kimi K3 achieved an Elo score of 1543 in the updated AA-Briefcase benchmark, trailing Fable5’s 1574 but outperforming GPT-5.6 Sol and Claude models. The test involved extracting data from over 2,000 emails and documents across four projects and 91 tasks. Kimi K3 completed 51% of objectives, scored 1754 in analytical quality, but underperformed in final presentation. Task costs increased tenfold to $10.57, with each run requiring 56.4 minutes and 120,000 tokens. Interest rate news had no direct impact on the results.

ChainThink reports that on July 22, Artificial Analysis updated the AA-Briefcase rankings, with Kimi K3 scoring 1543 Elo, just below Claude Fable 5’s 1574 and above GPT-5.6 Sol’s 1501, Claude Sonnet 5, and Claude Opus 4.8.

AA-Briefcase evaluation requires the model to locate information from nearly 2,000 emails, Slack messages, and company documents, and deliver tables, presentations, and interface prototypes, encompassing 4 long-term projects and 91 confidential tasks.

In detail, the objective pass rate for K3 is 51%, lower than Fable 5’s 56%; the analysis quality score is 1754, slightly higher than Fable 5’s 1744.

K3's main gap lies in final output quality, falling below GPT-5.6 Sol and Opus 4.8. In terms of cost, K3 averages $10.57 per task, approximately ten times that of Kimi K2.6;

Average of 83 rounds executed, outputting 120,000 tokens, taking 56.4 minutes—approximately 2.5 times longer than Fable 5 for similar tasks.

Source:Show original
Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.