QuantMLA INT4 cache doubles long-context attention throughput

QuantMLA's dual-path INT4/INT2 cache compresses the combined content-and-RoPE cache by 3.59x at a 128k-token window and eliminates memory preemptions. On long sequences it delivers over 5x the whole-job throughput of a BF16 baseline by avoiding capacity-driven recomputation rather than speeding up the attention kernel.

• 3.59x cache compression at a 128k-token window
• Over 5x whole-job throughput versus BF16 baseline
• Gain comes from avoiding recomputation, not kernel speedup
• Enables 8k+ context windows without breaching real-time SLAs