Moore Threads open-sources Megatron-MUSA-Titan for trillion-parameter MoE training on its GPUs

Moore Threads has published Megatron-MUSA-Titan on GitHub, a training stack for trillion-parameter mixture-of-experts models on clusters of more than 10,000 of its GPUs. It covers pretraining and reinforcement learning and runs upstream Megatron-LM code without modification.

• Targets 1T-parameter MoE models on clusters of 10,000+ Moore Threads GPUs
• Runs Megatron-LM, slime and miles code unchanged via torch_musa and MUSA C
• Supports blockwise FP8 training, DeepEP and compute-communication overlap
• Repo includes a Kimi-K2-style 1T MoE pretraining configuration