same model, same tasks, and one harness costs twice as much per solved task garry tan's running open source evals that swap only the wrapper around the model. on opus, openclaw pays about $0.44 per solve and ~99s a try. claude code $0.22 and 40s. on sol, codex was cheapest at $0.17 the scores didn't separate at all. 7 setups, 20 coding tasks, all landed 71-79%, and 14 of the 20 tasks came out identical everywhere. so the test was just too easy (harder terminal-bench rerun is going now) everyone arguing about which agent is smartest is arguing about 6 tasks right now. i'd pick the cheap one and wait for the rerun
Sad CreatorShare
Source:Show original
Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information.
Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.