DeepSeek V4.1 Flash 1017: two EXL3 quants, one runs on a single DGX Spark
The 1.61 bpw EXL3 quant fits one DGX Spark at ~112.9 GiB with 94.68% assistant-token top-1 agreement; the 2.50 bpw two-Spark version reaches 96.71%.
🚨 𝗧𝗪𝗢 𝗡𝗘𝗪 𝗗𝗘𝗘𝗣𝗦𝗘𝗘𝗞 𝗩𝟰.𝟭 𝗙𝗟𝗔𝗦𝗛 𝗥𝗘𝗟𝗘𝗔𝗦𝗘𝗦! 🛡️🔥 I’m excited to share TWO alternate DERISKED SAGE-EXL3 quantizations for cybersecurity research and security-focused AI workflows! These are the HIGHEST quality EXL3 quants you can find for this Deepseek model! Built in collaboration with @Blackfrost_AI 🥶❄️ 🟢 1.61 bpw — ONE DGX SPARK ✅ 94.68% assistant-token top-1 agreement ✅ ~112.9 GiB resident footprint 🟣 2.50 bpw — TWO DGX SPARKS ✅ 96.71% assistant-token top-1 agreement ✅ Higher precision for those with more memory - practically lossless 😎 Both measured against DeepSeek’s official FP8/FP4 checkpoint. These are token-fidelity results, not cybersecurity task scores. 𝗠𝗢𝗥𝗘 𝗖𝗛𝗢𝗜𝗖𝗘𝗦. 𝗠𝗢𝗥𝗘 𝗟𝗢𝗖𝗔𝗟 𝗔𝗜. 𝗠𝗢𝗥𝗘 𝗙𝗥𝗘𝗘𝗗𝗢𝗠 𝗧𝗢 𝗘𝗫𝗣𝗘𝗥𝗜𝗠𝗘𝗡𝗧! 🚀 🔗 1.61 bpw — ONE SPARK https://huggingface.co/vcruz305/DSV4.1-Flash-1017-DERISKED-EXL3-1.61bpw 🔗 2.50 bpw — TWO SPARKS https://huggingface.co/vcruz305/DSV4.1-Flash-1017-DERISKED-EXL3-2.50bpw Huge thanks to @Blackfrost_AI for not only making the model, but providing the compute for us to continue collaborating on these releases! 🙏🔥
This post quotes another post: “DeepSeek V4.1 Flash at 1.62bpw on one DGX Spark: 35.5 tok/s decode”