DeepSeek V4.1 Flash 1017 on four DGX Sparks: 81.8 tok/s, tool eval 93/100
TensorFold with EXL3 4.0bpw quant across four Sparks in TP4: 82.8 tok/s decode on code, 2,077 tok/s reading a 32K prompt, 15.8s first token; QA 90.7%, MMLU 93.7%, Tool Eval 93/100.
Community Day, post 5: the numbers. DeepSeek-V4.1-Flash-1017 on four DGX Sparks: 81.8 tok/s with @bertholomusai's TensorFold recipe. bertholomus' own EXL3 4.0 bpw quant, one model split across all four Sparks (TP4). The recipe as of Oct 22, commit 8f84e7a, with the author's sampling and context. My Sparks are cabled as a switchless ring, so it runs in the recipe's neighbor-ring mode, not the switched setup the author serves. One Spark, an ASUS GX10, has an older driver. Thinking off for every number here. Decode, code: 82.8 tok/s Decode, prose: 80.8 tok/s Reading a 32K-token prompt: 2,077 tok/s First token after a 32K prompt: 15.8 s Four requests at once: 116.8 tok/s in total Eight requests at once: 141.2 tok/s in total Context: 1,048,576 tokens, the recipe's setting Largest context I tested: 131,072 tokens The recipe runs 16 streams, so all eight fit at once. Quality and tool calls in Mia's sparkDash 2.50.0: QA: 90.7% ± 4.7 points MMLU: 93.7% ± 2.9 points Tool Eval: 93/100 Open: 4 of 150 QA answers hit the token limit and count as wrong. Every number, the exact command and the raw output: https://niklaslenz.com/v2/deepseek-v41-flash-4sparks-bertholomus-tp4/ You can play the world it built there in your browser. Recipe, quant and TensorFold fork by @bertholomusai. TensorFold engine by @ashxhart. sparkDash by @MiaAI_lab. Model by DeepSeek. My pick on 4 Sparks: DeepSeek-V4.1-Flash-1017 on bertholomus' recipe.