DeepSeek V4.1 Flash 1017 EXL3 3.5bpw quantization: 240 GiB, 0.8%/0.1% degradation
A 3.5bpw EXL3 quant (240 GiB) of DeepSeek V4.1 Flash 1017 scores only 0.8% / 0.1% worse than base on code and prose sets, with 2.9bpw numbers and Kit KMAP_HOST setup included.
Now at 3.5 bpw too: 240 GiB. Same sets, code / prose: base 1017: 1.8749 / 6.4277 3.5 bpw: 1.8895 / 6.4341 (+0.8% / +0.1%) 2.9 bpw: 1.9094 / 6.4520 Kit users: point KMAP_HOST at this repo's exl3_k_map.json. 🤗 https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-3.5bpw
1017, part 2: EXL3. DeepSeek-V4.1-Flash-1017 at 2.9 bpw for exllamav3: 196 GiB of weights. The first EXL3 of 1017 on Hugging Face. Perplexity vs the base model: +1.8% on code, +0.4% on prose. 🤗 https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw
This post quotes another post: “First NVFP4 quant of DeepSeek V4.1 Flash 1017 lands on Hugging Face”
