Huawei details Stellar AI Fabric design to ease KV Cache bottleneck

At the 2026 Advanced Storage Conference in Shanghai, Huawei outlined its Stellar AI Fabric with the Hyper-Converged Fabric architecture for moving KV Cache between compute, storage and network. In its own lab tests on GLM-5.2, direct KV Cache offloading delivered 7.3x the inference TPS of a conventional memory-based approach, with coordinated IO scheduling adding another 20%.

• Direct KV Cache offloading delivered 7.3x inference TPS in Huawei lab tests
• Storage-network coordinated IO scheduling added a further 20% efficiency
• The HCF network layer lifts storage IO performance by more than 30%
• KV Cache for an inference cluster can reach petabyte scale