DeepSeek V4.1 Flash on two DGX Sparks: 68.5 tok/s with EXL3 2.9bpw
Running V4.1 Flash on two DGX Sparks with Mia's EXL3 2.9bpw quant: 68.5 tok/s decode (72 code, 65 prose), 2124 tok/s on a 32K prompt, 94% QA and 93/100 tool eval.
Community Day, post 7: the numbers. DeepSeek-V4.1-Flash-1017 on two DGX Sparks: 68.5 tok/s on @MiaAI_lab's latest 2-Spark recipe. Her EXL3 2.9 bpw quant on TensorFold, one model split across both Sparks. Her recipe as of Oct 21, commit 370daf0, with her sampling and the context she measures with. Thinking off for every number here. Decode, code: 72.0 tok/s Decode, prose: 65.0 tok/s Four requests at once: 98.4 tok/s in total Eight requests at once: 99.6 tok/s in total Reading a 32K-token prompt: 2,124 tok/s Context: 524,288 tokens Quality and tool calls in Mia's sparkDash 2.50.0: QA: 94.0% ± 3.9 points MMLU: 91.9% ± 3.2 points Tool Eval: 93/100 The recipe runs four streams, so at eight requests the other four wait their turn. Same two Sparks, her previous version from Oct 19: Decode, code before: 73.0 tok/s Decode, code now: 72.0 tok/s Decode, prose before: 61.7 tok/s Decode, prose now: 65.0 tok/s Code stays the same, prose got about 5% faster. sparkDash QA, MMLU and Tool Eval came out exactly the same. Every number, the exact command and the raw output: https://niklaslenz.com/v2/deepseek-v41-flash-2sparks-mia-tensorfold/ You can play the world it built there in your browser. Recipe, EXL3 quant and sparkDash by @MiaAI_lab. TensorFold engine by Ash Hart, its DeepSeek-V4.1 support by BertholomusAI, model by DeepSeek. My pick on 2 Sparks: DeepSeek-V4.1-Flash-1017 on Mia's recipe.