DeepSeek open-sources DeepInfer engine, minus cache and load balancing

DeepSeek released DeepInfer, the inference engine behind its API, as open source, but its README states that the KV cache, load balancing and prefill/decode disaggregation are not included. DeepSeek pegs revenue for such a service at $38 per GPU-hour, and the smallest efficient deployment needs 128 GPUs.

• 4 of 5 routing and cache components from DeepSeek's diagram are missing
• Smallest efficient deployment: 64 GPUs for prefill plus 64 for decode
• Cache sized at 14.4 TB per GPU, retaining KV cache over 72 hours
• Only 6.82% of code matches vLLM; Huawei kernels are 52% larger than Nvidia's