蚂蚁与 vLLM 开源 Humming 量化内核：V4.1 Flash 吞吐提升 14%–43%
在 8 张 H20 上把 V4.1 Flash 的 MoE 层从 Marlin W4A16 换成 Humming W4A8，端到端吞吐提升约 14%–43%，随 vLLM 0.29.0 起提供。
Why does a 4-bit model sometimes run slower than you expected? Often it comes down to the kernels. Ant Group and the vLLM community just open-sourced Humming, one kernel library for quantized inference across many weight and activation formats, dense and MoE, on NVIDIA GPUs from Turing onward. Kernels are compiled on demand from shared CUDA components. In their DeepSeek-V4.1-Flash test on eight H20 GPUs, switching the MoE layers from Marlin W4A16 to Humming W4A8 raised end-to-end throughput by about 14% to 43%. It ships with vLLM 0.29.0 and later. If you serve a quantized MoE model, try the Humming backend on a copy of your real traffic and compare time to first token along with throughput. #PyTorch Link in Comments