DeepSeek V4.1 Flash on four DGX Sparks: 70.2 tok/s decode
Hands-on test running V4.1 Flash across four DGX Sparks with TensorFold TP4: 70.2 tok/s prose decode, 69.5 tok/s code, 3,743 tok/s on a 32K prompt, and 139 tok/s aggregate at 8 requests.
I interrupted every run and got straight to it: DeepSeek-V4.1-Flash-1017 on four DGX Sparks: 70.2 tok/s on prose with @MiaAI_lab's TensorFold recipe. Setup: – One model split across all four Sparks (TP4), the FP8/MXFP4 checkpoint as DeepSeek published it – The recipe as of Oct 25, commit 6beaad9, with the 8 TF_DS_* switches her README table sets on for her cluster, and her server defaults for context and streams – My Sparks are cabled as a switchless ring, the setup her numbers use – I built the recipe's image offline from the same pinned sources – Thinking off for the speed-test numbers below Speed: – Decode, code: 69.5 tok/s – Decode, prose: 70.2 tok/s – 500 words: about 10.0 s – Reading a 32K-token prompt: 3,743 tok/s – First token after a 32K prompt: 8.8 s – DSpark drafts accepted: 57.3% – 4 requests at once: 115.9 tok/s in total – 8 requests at once: 139.0 tok/s in total – Context: 1,048,576 tokens, the recipe's setting Good to know: – The recipe runs 16 streams, so all eight run at once. – With her method rebuilt from her README: code 119.9 tok/s (hers 117.4), code at eight requests 206.6 (hers 183.7). – Prompt reading at 32K: 3,872 tok/s with my 2200 MHz clock cap, 4,019 without it (hers 4,127). Prose at eight requests 157.0 (hers 165.3); her test prompts aren't published, so I used my own. – My speed test uses the model card's sampling, so its decode numbers differ from her README's method. The voxel is building right now and probably finish within the next 90 minutes. Mia's recipe and the exact setup: https://mia-ai.net/models/DeepSeek-V4.1-Flash-1017-4x-DGX-Sparks-TensorFold Recipe by @MiaAI_lab. TensorFold engine by @ashxhart. Model by @deepseek_ai.
This post quotes another post: “DeepSeek V4.1 Flash official weights on 4x DGX Spark: 66 tok/s single stream, 173 tok/s at 16 concurrent”