Nhà phát triển mở nguồn C Engine để chạy Kimi K3 với 2,78T tham số trên 8GB RAM

The Huoxing Finance report states that on August 8, a developer recently open-sourced the project kimi-k3-in-c, attempting to run the Kimi K3 model with 2.78 trillion parameters on a device with only 8GB of RAM. The project, just 176KB in size, is written purely in C99 and does not rely on GPU, CUDA, PyTorch, or BLAS—achieving model inference using only the CPU. This solution leverages the MoE (Mixture of Experts) architecture of Kimi K3. Although the model has a total of 2.78T parameters, only 16 out of 896 experts per layer are activated. Therefore, the developer did not load the full model weights of approximately 1.56TB into memory; instead, most expert weights are stored on NVMe storage and loaded in real time according to inference demands. Meanwhile, certain dense layers (dense trunk) are also loaded in a streaming, layer-by-layer manner. However, this solution still has significant performance limitations. Under the 8GB RAM mode, generating a single token takes approximately 32.7 seconds and requires nearly 1.7TB of high-speed storage. The developer notes that this approach is currently more of an experimental exploration into optimizing large model inference infrastructure and lacks practical production value. Nevertheless, its method of “streaming loading from disk + sparse activation via MoE” offers a novel direction for enabling cost-effective deployment of ultra-large models in the future.