Hardware2 hOctober 20, 2026, 00:38

d-Matrix halves AI token latency with hybrid GPU-XPU inference at OCP

At the OCP Global Summit in San Jose, d-Matrix ran a live GPT-OSS-20B inference pipeline that routed attention layers to NVIDIA H100 GPUs and expert FFN layers to its Corsair XPUs. Time per output token fell from 2.59 ms to 1.28 ms, a 50.6% reduction at 1,000-token context.

Tech Times

Open source

d-Matrix halves AI token latency with hybrid GPU-XPU inference at OCP

Time per output token dropped from 2.59 ms to 1.28 ms, a 50.6% cut
Corsair SRAM delivers 150 TB/s versus 3.35 TB/s HBM3 on H100
Each Corsair card holds 4 GB SRAM; a SquadRack carries 256 GB
AFD benefits only MoE models under strict latency SLOs

#D-Matrix#Nvidia#Corsair#GPT-OSS-20B

d-Matrix halves AI token latency with hybrid GPU-XPU inference at OCP

d-Matrix unveils Squadron multi-rack AI inference clusters

BesselAI Fluid Computing Cuts AI Inference Latency 60%, Targets 200K GPUs

SenseNova: robot benchmarks ignore inference latency

TokenRouter: token-level LLM routing cuts inference cost 18% and latency 31%