ByteDance Seed Paper Finds 'Phase Sensitivity' Blind Spot in Chunked KV-Cache Compression

ByteDance Seed researchers identified a periodic weak spot in models that compress their KV cache in fixed-size chunks, a design DeepSeek-V4 uses. On a 128K-token needle-in-a-haystack task, base checkpoint accuracy differed by up to 40.2 percentage points depending on the target's phase.

• Base checkpoint accuracy differed by up to 40.2 percentage points depending on target phase
• Tests covered DeepSeek-V4-Flash, V4-Pro and V4.1-Flash at 128K tokens
• V4.1-Flash with stride 2 showed the smallest gap, but periodicity remained
• Authors advise evaluating such models by shifting input a few tokens