SGLang runs DeepSeek's own DeepInfer benchmarks and matches the engine

The SGLang team published measurements of its inference engine against DeepInfer, DeepSeek's open-sourced serving engine, using DeepInfer's own benchmark workload on identical hardware. It reports up to 1.04x prefill throughput on 4x GB300 and 4x B300 and 1.1–7.4% shorter decode steps on 64x GB300. SGLang now also runs DeepSeek's kernels, including FlashMLA, DeepGEMM, DeepEP, DeepSelect, TileLang and the Mega series.

DEV Community

Open source

SGLang runs DeepSeek's own DeepInfer benchmarks and matches the engine

Up to 1.04x prefill throughput on 4x GB300 and 4x B300
Decode steps 1.1–7.4% shorter on 64x GB300
SGLang integrates FlashMLA, DeepGEMM, DeepEP and Mega kernels
Measurement methods differ: DeepInfer times ~9 batches on GPU 0

#SGLang#DeepSeek#DeepInfer#Nvidia

SGLang runs DeepSeek's own DeepInfer benchmarks and matches the engine

DeepSeek open-sources DeepInfer, the engine behind its API

DeepInfer beats vLLM on throughput with 8 B200 GPUs

Four local LLMs up to 748B parameters benchmarked on a single RTX 5090

DeepSeek-V4.1-Flash (763B) runs on a single 96 GB GPU with three key settings