ME News report, October 9 (UTC+8), Beating AI Bulletin: ByteDance’s Seed team discovered a periodic weakness in DeepSeek-V4’s long-text processing. Researchers prompted the model to complete a segment of code, keeping the code itself unchanged while only extending a preceding block of irrelevant comments; the model’s responses alternated between correct and incorrect outputs. The base V4-Flash model exhibited this pattern every 4 tokens, while V4.1-Flash showed a similar phenomenon with a cycle of 2 tokens. The issue is linked to DeepSeek’s memory-saving “chunked KV cache compression,” which consolidates information from consecutive tokens to reduce computational costs when processing long texts. However, this compression results in uneven retention of information across different positions, making certain content harder to accurately retrieve later. ByteDance refers to this periodic retrieval discrepancy as “phase sensitivity.” In retrieval tests involving approximately 128,000 tokens, the base V4-Flash model showed a maximum accuracy difference of 40.2 percentage points across different positions. After post-training, this gap narrowed to 19.1 percentage points; for V4.1-Flash, it further decreased to 6.1 percentage points, though the periodic fluctuations persisted. ByteDance also trained multiple control models from scratch and found that the fluctuation cycle matched the compression stride—models without chunked compression showed no such pattern. (Source: BlockBeats)
ByteDance Identifies Periodic Issues in DeepSeek-V4's Long-Text Processing
KuCoinFlashShare
Summary
Value investing in crypto is facing new scrutiny after ByteDance’s Seed team discovered periodic errors in DeepSeek-V4’s long-text processing. When given the same code with extended comments, the model alternates between correct and incorrect outputs. The issue arises from DeepSeek’s chunked KV cache compression, which merges tokens to conserve memory but results in inconsistent data retention. In 128,000-token tests, V4-Flash’s accuracy varied by up to 40.2 percentage points. Post-training reduced this to 19.1, and further to 6.1 in V4.1-Flash, though periodic inconsistencies persist.
Source:Show original
Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information.
Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.