4Paradigm and RiseUnion open-source ModelSphere for heterogeneous LLM serving

4Paradigm and RiseUnion have open-sourced ModelSphere under Apache 2.0, a production platform for LLM deployment covering cluster routing, a shared KV cache pool, SLO-driven autoscaling and online AutoTune. Its EngineX layer schedules Huawei Ascend, Iluvatar, Moore Threads and Hygon DCU accelerators in one fleet. The companies claim up to 285% higher peak throughput versus an internal baseline.

• Apache 2.0 license, code on GitHub at modelsphere/modelsphere
• Adaptations for DeepSeek-V4 and V4-Flash, Qwen, Kimi, MiniMax, GLM
• EngineX schedules Ascend, Iluvatar, Moore Threads and Hygon DCU
• Claimed 285% gain in peak throughput over internal baseline