Developer Opens Source C Engine to Run 2.78T-Parameter Kimi K3 on 8GB RAM

Huo Xing Caijing reports that on August 8, a developer recently open-sourced the project kimi-k3-in-c, attempting to run the Kimi K3 model with 2.78 trillion parameters on a device with only 8GB of RAM. The project, just 176KB in size, is written entirely in pure C99 and does not rely on GPU, CUDA, PyTorch, or BLAS—performing model inference solely via CPU. This approach leverages the MoE (Mixture of Experts) architecture of Kimi K3. Although the model has a total of 2.78T parameters, only 16 out of 896 experts per layer are activated at any time. Therefore, the developer did not load the full ~1.56TB of model weights into memory; instead, most expert weights are stored on NVMe storage and loaded on-demand during inference. Additionally, certain dense layers (dense trunk) are loaded in a streaming, layer-by-layer manner. However, this solution still has significant performance limitations. Under the 8GB RAM configuration, generating a single token takes approximately 32.7 seconds and requires nearly 1.7TB of high-speed storage. The developer notes that this approach is currently more of an experimental exploration into optimizing large model inference infrastructure rather than a practical production solution. Nevertheless, its use of “streaming disk loading + sparse MoE activation” offers a novel direction for enabling cost-effective deployment of ultra-large models in the future.