Meta AI Loop Scaling Laws: Half-Size Looped MoE Matches Larger Models

Researchers at Meta AI published the first mathematical framework governing both recurrent looping and Mixture-of-Experts sparsity in LLM design. At matched training compute, a Looped MoE configured with their laws matches the reasoning performance of a conventional MoE roughly twice its size.

• Loop Scaling Laws predict held-out loss more accurately across model sizes
• The laws validate at trillion-token training scale
• Zero recurrence recovers standard MoE scaling laws as a special case
• Looped MoE matches reasoning of a conventional MoE about twice its size