NVIDIA QAD-quantized GLM beats DeepSeek V4.1 Flash EXL3 in local test
On two 128GB DGX Sparks, the author found NVIDIA's QAD-distilled GLM-5.3-Flash-FP4 clearly better than the DeepSeek-V4.1-Flash 2–3 bpw EXL3 builds he tested for coding and math.
This is why NVIDIA’s QAD quants matter for local AI. GLM-5.3-Flash-FP4-Local gets extra training to recover what compression loses. With quantization-aware distillation (QAD), the original BF16 model acts as a teacher, helping the compressed model learn to give similar answers while working with lower-precision weights. That takes more work than a regular quant conversion: NVIDIA used 500 million training tokens over 3 epochs for this release. The result is a 320B model using about 166 GiB, validated on two 128 GB DGX Sparks, with room for context. Quality stays close to BF16 on NVIDIA’s two reported benchmarks: GPQA 91.3% vs 92.1%; tool use 92.4% vs 91.9%. EXL3 at 2–3 bits per weight can squeeze models into even less memory. But fitting a bigger model doesn’t automatically mean better answers. For my coding and math work, this QAD GLM build is a much better choice than the DeepSeek-V4.1-Flash 2-3 bpw EXL3 builds I tested on the same two Sparks. I’d rather preserve more of a model’s ability than chase parameter count at the lowest possible bitrate. More releases like this, please @NVIDIAAI !!!! Model: https://huggingface.co/nvidia/GLM-5.3-Flash-FP4-Local How QAD works: https://developer.nvidia.com/blog/how-quantization-aware-training-enables-low-precision-accuracy-recovery/