Adaptive compute cuts AI inference latency by up to ~45%

Researchers described three training-free primitives for faster inference: ALLOT reallocates block-sparse attention in video diffusion, Lafa speeds up flow-based LLMs with block drafts, and MemT-1.0-Nano shrinks long-context memory. Combined, they promise roughly a 45% latency reduction.

• ALLOT delivers 1.90× speedup and a 3.38 dB PSNR gain on HunyuanVideo 13B
• Lafa with block size four is about twice as fast while more accurate on average
• MemT-1.0-Nano gives 1.86× speedup and 66× less storage for 128K-token histories
• Gains depend on sparsity density and context length