NoteBank: replaying saved snippets restores long-context recall in linear-attention models

KAUST researchers proposed NoteBank, a training-free add-on for linear-attention and state-space models like Gated DeltaNet and Mamba-2 that stores a few verbatim text snippets and replays them when a question arrives. On six RULER needle-in-a-haystack tasks at one million tokens, GDN's mean accuracy rose from 1.7% to 70.3%, with storage about 1/2,600 of a full KV cache.

• GDN accuracy on six RULER tasks at 1M tokens rose from 1.7% to 70.3%
• Storage is 74.5 MiB versus 192 GiB for a full KV cache at matched settings
• Extra compute is at most 0.025% of analytical FLOPs
• Gains also reported on LongBench v1/v2 and MemoryAgentBench across 1.3B-9B models