deepseek-v4.1POST
vLLM and Ant's Humming: DeepSeek V4.1 Flash throughput +14% to +43% on 8x H20
vLLM and Ant Group open-sourced JIT-quantized GEMM kernels called Humming, lifting V4.1 Flash throughput on 8x H20 by 14% at concurrency 1 and 43% at 64.
海猿
@guolinwangArchived 10 days agoOriginally posted 2026.10.22
海猿 @guolinwangOriginally posted 2026.10.22
vLLM + Ant Group open-sourced Humming, JIT quantized GEMM kernels. DeepSeek-V4.1-Flash on 8x H20, same MXFP4 weights, FP8 activations: +14% throughput at concurrency 1, +43% at 64. Big batches are compute-bound, so 4-bit weights alone stop paying. https://vllm.ai/blog/2026-10-21-humming
x.com/guolinwang/status/2113030967929520346Archived 2026.10.22 12:01