deepseek-v4.1POST
DeepSeek V4.1 Flash 1017 on two DGX Sparks: EXL3 2.5bpw hits ~107 tok/s on code
An EXL3 2.5bpw quant (~185GB) with TensorFold TP2 runs 1M context on two DGX Sparks at ~107 tok/s on code and ~2.4K tok/s prefill, but tool calls degrade most at 2.5bpw, so 4 Sparks at 4.0bpw is still recommended for agents.
𝙒𝙞𝙡𝙘𝙤 𝘷𝘢𝘯 𝘽𝙤𝙭𝙩𝙚𝙡 ⚡🚀🧲🌞
@nuonrgArchived 8 days agoOriginally posted 2026.10.24
𝙒𝙞𝙡𝙘𝙤 𝘷𝘢𝘯 𝘽𝙤𝙭𝙩𝙚𝙡 ⚡🚀🧲🌞 @nuonrgOriginally posted 2026.10.24
DeepSeek V4.1 Flash 1017 now runs on just 2 DGX Sparks: @bertholomusai's EXL3 2.5bpw quant (~185GB) + TensorFold TP2, 1M context, ~107 tok/s on code, ~2.4K tok/s prefill. Tool calls lose the most at 2.5bpw, so 4 Sparks at 4.0bpw is still the pick for agents. https://huggingface.co/bertholomus/DeepSeek-V4.1-Flash-1017-EXL3-2.5bpw
x.com/nuonrg/status/2113734648391954770Archived 2026.10.24 12:07