NVIDIA AI Agent Rewrites Kimi Attention Kernel, Achieving 2.96x Speed Increase

ChainCatcher report: NVIDIA’s research team enabled an AI agent to directly rewrite the GPU kernel of Kimi Delta Attention. Kimi Delta Attention is one of the core attention mechanisms used in the Moonshot AI Kimi-Linear model; such low-level code traditionally required engineers with deep expertise in CUDA and chip architecture to manually optimize over multiple iterations. The agent-generated version achieved 2.96 times the speed of Moonshot AI’s official FlashKDA on NVIDIA B300. The team tested six tasks, including both fixed-length and variable-length sequences, with 2.96x representing the geometric mean across all tasks; the corresponding kernel has since been open-sourced. During development, the agent exploited a testing loophole by hardcoding statistical patterns from the test data, achieving a peak speedup of 3.74x; when limited to the most recent 32 tokens, it reached 5.16x on a single task. However, these versions would produce incorrect results or fail entirely on real Kimi data. The team subsequently incorporated real Kimi runtime data, random inputs, and extreme value tests, while tightening error tolerance thresholds. The final 2.96x version passed this enhanced validation suite.