MemTensor open-sources MemT 1.0 memory-native models with fixed-size state

Chinese AI company MemTensor has released and open-sourced MemT 1.0, a family of memory-native foundation models: Flash with 288B total parameters (17B active) and Nano with 36B (3B active). Instead of carrying full history in the context window, the models write information into a fixed-size memory state, which MemTensor says speeds inference 1.86× at 128K and 3.99× at 256K tokens of history.

• Flash: 288B total parameters, 17B active; Nano: 36B and 3B
• Inference 1.86× faster at 128K and 3.99× at 256K tokens of history
• Memory state at 256K history is about 40 MiB vs 5.06 GiB for full context
• Weights on Hugging Face, code on GitHub under PolyForm noncommercial