Fable 5 fails the most challenging tasks in AI agent benchmark, costing 4 to 12 times more than competitors.

ME AI News: According to monitoring by Beating, a project led by UC Berkeley’s RDI in collaboration with hundreds of industry experts, has launched a new benchmark for evaluating AI agents: Agents' Last Exam (ALE), designed to assess agents’ ability to perform real-world digital professional tasks. ALE covers 55 digital professional subdomains and includes over 1,500 validated tasks derived from actual human expert projects, supporting result verification in both GUI and CLI environments. Initial tests included cutting-edge systems such as Fable 5, GPT-5.5, and Composer 2.5. According to the latest official comparison metrics, in the most challenging tasks requiring sustained reasoning and deep expertise, all tested agents achieved a 0% success rate—Fable 5, just released this week, also scored zero. This is primarily due to safety protocols triggering: approximately 35% of Fable 5’s tasks were rolled back to run on the older Opus 4.8 system, significantly undermining its overall performance compared to other benchmarks. In terms of single-task API cost, Fable 5 averages $15.70, far exceeding GPT-5.5’s $3.80 and Composer 2.5’s $1.33—resulting in 4 to 12 times higher expenses for the same tasks. The evaluation also revealed that the most common failure mode among agents is prematurely declaring success without verifying outcomes, often missing files or miscalculating data. For CLI-based agents, the evaluation team has simultaneously released ALE-CLI, a subset of the benchmark. Compared to existing benchmarks like Terminal-Bench and SWE-bench-Pro, ALE-CLI spans 40 subdomains, with human average task completion times ranging from hours to weeks. In CLI evaluations, even the best-performing agent achieved only a 25.2% pass rate. The evaluation team notes that while the era of usable AI agents has arrived, there remains a long way to go before they can reliably replace human professionals. (Source: MLion)