vLLM matches DeepSeek's DeepInfer engine on half the GPUs
After DeepSeek open-sourced DeepInfer, the engine serving V4.1 Flash, a team got vLLM to match it on the official decode config using half the GPUs, then scaled to 954K tokens/s per GPU across a GB300 NVL72 rack.
DeepSeek open-sourced DeepInfer this week, the engine behind its own DeepSeek-V4.1-Flash serving. We think it is the most useful kind of release: a purpose-built engine, originally built on @vllm_project, that shows what one model can do on one piece of hardware. Our team got vLLM to match it on DeepSeek's own decode configuration, using half the GPUs, and then kept going to where an offline reference engine does not reach: online serving across a full GB300 NVL72 rack, at 954K tokens/s per GPU on SemiAnalysis AgentX and $38.96 of token value per GPU-hour. Our view is that the gap between a lab’s reference engine and the engine you run in production should be close to zero, and that closing it is mostly careful systems work rather than new algorithms.
This post quotes another post: x.com/1774187564276289536/status/211405427737964…
The same engineering runs on the Inferact platform. We serve leading open models like DeepSeek-V4.1-Flash on your hardware or ours, tuned for your workload, with the vLLM maintainers on the other end. https://inferact.ai