V4.1 Flash EXL3 on two DGX Sparks: 107 tok/s across 4 lanes
Running V4.1-Flash-1017 with the TensorFold engine and MIA AI Lab's EXL3 quants on two DGX Sparks gave 107 tok/s aggregate over 4 concurrent lanes, about 28 tok/s each, which the author calls very usable desktop-level serving.
Happy to report: TensorFold engine (@ashxhart) plus the MIA AI Lab recipe (@MiaAI_lab) for 2 x DGX Spark boxes, DeepSeek-v4.1-Flash-1017, EXL3 quants published by Mia, confirmed running 4 concurrent lanes at aggregate 107 tokens/second using unique prompts with sustained long-term decoding. This is an extraordinary result. Each lane is nearly 28 tokens/second, which is very usable. I didn't measure prefill. Given the strength of this remarkable DeepSeek model, to have it within reach of your desktop with 2 spark boxes is really an achievement. Those 2 sparks used to cost just $8000 total. Now they about $16K total, and it's because of these use cases making the hardware more valuable by the day! Nvidia should be thanking MIA AI Lab and Ash Hart, and many others working in this open source space. They are the ones that make Spark boxes FAR more valuable! See my Decentralize TV interview with Ash Hart! Link in the comments.