双 DGX Spark 跑 V4.1 Flash EXL3:4 路并发合计 107 tok/s
用 TensorFold 引擎加 MIA AI Lab 的 EXL3 量化配方,在两台 DGX Spark 上跑 V4.1-Flash-1017,4 路并发合计 107 tok/s、每路约 28 tok/s,作者称桌面级部署已很实用。
Happy to report: TensorFold engine (@ashxhart) plus the MIA AI Lab recipe (@MiaAI_lab) for 2 x DGX Spark boxes, DeepSeek-v4.1-Flash-1017, EXL3 quants published by Mia, confirmed running 4 concurrent lanes at aggregate 107 tokens/second using unique prompts with sustained long-term decoding. This is an extraordinary result. Each lane is nearly 28 tokens/second, which is very usable. I didn't measure prefill. Given the strength of this remarkable DeepSeek model, to have it within reach of your desktop with 2 spark boxes is really an achievement. Those 2 sparks used to cost just $8000 total. Now they about $16K total, and it's because of these use cases making the hardware more valuable by the day! Nvidia should be thanking MIA AI Lab and Ash Hart, and many others working in this open source space. They are the ones that make Spark boxes FAR more valuable! See my Decentralize TV interview with Ash Hart! Link in the comments.