Reward shaping cuts LLM reasoning traces by 65%
The RLCD² method reshapes reinforcement-learning rewards to reactivate a dormant THINK mode in hybrid models Qwen3 and StartLux-Decision. It raises the share of traces finishing within the generation limit by up to 48 points and cuts average trace length by 65% while largely preserving accuracy; a 9B model's termination rate rose from 45.9% to 93.9%.
- For the 9B model, mean trace length fell from 2,494 to 868 tokens
- Termination rate rose from 45.9% to 93.9%
- THINK win rate jumped from 50% to 94% at the 8B checkpoint
- Patching the think subspace cut accuracy to 0.244, removing 75% of the RL gain
Read next
AI