deepseek-v4.1POST
SGLang optimizes V4.1 Flash: 538→943 tok/s per user, 348K→654K tok/s per GPU
The SGLang team says MegaKernel lifts interactivity to 943 tok/s per user, while WideEP and system optimizations raise total throughput per GPU from 348K to 654K tok/s.
SGLang
@sgl_projectArchived 2 days agoOriginally posted 2026.10.27
SGLang @sgl_projectOriginally posted 2026.10.27
We’ve been working to push both ends of the curve on DeepSeek V4.1 Flash 🚀 • For low latency, our MegaKernel optimizations take interactivity from 538 to 943 tok/s/user. • For high throughput, WideEP and system-wide optimizations take total throughput (cache + new input + output) from 348K to 654K tok/s per GPU. There’s still lots of room for improvement. We’ll share more updates soon.
This post quotes another post: x.com/semianalysis_/status/2114779348838338719
x.com/sgl_project/status/2114846465399144694Archived 2026.10.27 12:09