vLLM 用一半 GPU 追平 DeepSeek 自研 DeepInfer 推理引擎
DeepSeek 开源服务 V4.1 Flash 的 DeepInfer 引擎后，团队让 vLLM 在官方解码配置下用一半 GPU 追平，并在 GB300 NVL72 整机架做到每 GPU 954K tokens/s。
DeepSeek open-sourced DeepInfer this week, the engine behind its own DeepSeek-V4.1-Flash serving. We think it is the most useful kind of release: a purpose-built engine, originally built on @vllm_project, that shows what one model can do on one piece of hardware. Our team got vLLM to match it on DeepSeek's own decode configuration, using half the GPUs, and then kept going to where an offline reference engine does not reach: online serving across a full GB300 NVL72 rack, at 954K tokens/s per GPU on SemiAnalysis AgentX and $38.96 of token value per GPU-hour. Our view is that the gap between a lab’s reference engine and the engine you run in production should be close to zero, and that closing it is mostly careful systems work rather than new algorithms.
这条是引用着另一条写的：x.com/1774187564276289536/status/211405427737964…
The same engineering runs on the Inferact platform. We serve leading open models like DeepSeek-V4.1-Flash on your hardware or ours, tuned for your workload, with the vLLM maintainers on the other end. https://inferact.ai