DeepSeek-V4.1-Flash (763B) runs on a single 96 GB GPU with three key settings
The open-source deepseek-one-gpu engine serves the Q2_K-quantized DeepSeek-V4.1-Flash (264.5 GB) on one RTX PRO 6000 Blackwell with 96 GB of VRAM plus server RAM. It hits 63.11 tokens/s for one user and 152.53 tokens/s total across eight.
- 5,825 hottest experts (69.5 GiB) stay in VRAM, the other 9,535 live in RAM
- Plan for about 280 GB of RAM; the planner refuses hosts below the minimum
- Binding to the GPU's NUMA node lifted speed from 53 to 63 tokens/s
- HumanEval: 0.890 for the engine vs 0.982 for the full model via DeepSeek's API
Read next
AI