DeepSeek Plans to Train an 8-Trillion-Parameter Model, Three Times Larger Than the Current Largest Open Model

iconKuCoinFlash
Share
AI summary iconSummary
AI + crypto news: DeepSeek CEO Liang Wenfeng told investors the company is training a 2-trillion-parameter model and aims to develop an 8-trillion-parameter model. The current V4-Pro has 1.6 trillion parameters. The 8-trillion-parameter model would surpass Moonshot AI’s Kimi K3, which has 2.8 trillion parameters. Kimi K3 uses MoE, activating only 104 billion parameters per token. Crypto news: parameter count does not equate to active usage in every inference.
ME AI News, Dongcha Beating AI Brief: DeepSeek is planning to scale its model even further. CEO Liang Wenhong revealed to investors that the company is currently training a 2-trillion-parameter model, with plans to subsequently develop an 8-trillion-parameter model. The current flagship, V4-Pro, has only 1.6 trillion parameters. What does 8 trillion mean? Among publicly available large models, Moonshot AI’s Kimi K3, with 2.8 trillion parameters, is currently the world’s largest open-source model. DeepSeek’s planned 8-trillion-parameter model would be nearly three times larger. However, more total parameters don’t mean all parameters are activated during each inference. Kimi K3 uses a MoE architecture, activating only 104 billion parameters per token out of its 2.8-trillion total. (Source: BlockBeats)
Source:Show original
Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.