CoinDesk reports that on April 22 (UTC+8), according to monitoring by Beating, Princeton PhD student Yifan Zhang updated technical details of DeepSeek V4 on X. On April 19, he previewed “V4 next week” and listed three architectural components; tonight, he released the full parameter table and revealed for the first time the existence of a lightweight variant, V4-Lite, with 285B parameters. The total parameters of V4 amount to 1.6T. The attention mechanism is DSA2, combining DeepSeek’s previously used DSA (DeepSeek Sparse Attention) from V3.2 and the newly proposed NSA (Native Sparse Attention) from this year’s paper, with a head-dim of 512, paired with Sparse MQA and SWA (Sliding Window Attention). The MoE layer consists of 384 experts, with 6 activated at a time, using the Fused MoE Mega-Kernel. Residual connections continue to use Hyper-Connections. Training-related details disclosed for the first time include: the optimizer is Muon (a matrix-level optimizer applying Newton-Schulz orthogonalization to momentum updates), a pre-training context length of 32K, and GRPO used in the reinforcement learning phase with KL divergence correction. The final context length has been extended to 1M. The model is text-only. Zhang is not affiliated with DeepSeek, and DeepSeek has not responded to the above information.
DeepSeek V4 Technical Details Revealed: 1.6T Parameters, 384 Experts with 6 Activated
币界网Share
Summary
On-chain news broke on April 22 (UTC+8) when Princeton PhD student Yifan Zhang shared DeepSeek V4’s full specifications on X. V4 features 1.6 trillion parameters, a DSA2 attention mechanism, 384 MoE experts with 6 active per step, and a 1 million token context window. Zhang is not affiliated with DeepSeek, which has not commented. The network upgrade details suggest significant performance improvements.
Source:Show original
Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information.
Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.