Continual-learning anchors boost memorization 28-fold
A continual-learning method combining data, function and weight anchors with merged low-rank LoRA updates raised 100-task retention from 1.2% to 34.9%, a 28-fold gain. Naive sequential fine-tuning forgets almost immediately, with a memory half-life of just one task on Symbol-QA and LLM-QA.
- Retention after 100 tasks rose from 1.2% to 34.9%
- Naive fine-tuning loses memory after just 1–2 tasks
- Best result comes from data anchor plus merged LoRA
- Method tested only on three 100-task suites
Read next
AI