New benchmark: "StarCraft: Brood War Benchmark"
Whenever a new frontier model is released, YouTubers often upload content about it—you may have seen some of these videos.
This benchmark directly evaluates AI models’ performance in playing StarCraft: Brood War, and reportedly, none of the tested models surpassed beginner-level play.
Distinct playing styles emerged across models: CodeX favored unconventional, trick-based strategies over standard play; Claude played a more classical, textbook style; and Grock simply stopped playing mid-match.
All model-inference configuration combinations were pitted against each other in a round-robin format. For full details, please refer to the original report.
🥹 Leaderboard
1. Astra (xhigh) – 18 wins, 0 losses
2. Astra (medium) – 16 wins, 2 losses
3. Fable – 15 wins, 3 losses
4. Astra (low) – 14 wins, 4 losses
5. Sol (medium) – 13 wins, 5 losses
...