双 DGX Spark 上 2.9bpw 量化跑 V4.1 Flash：单流 73.5 tok/s，百万 token 三

Yume_X @yume_arasaki原推发布于 2026.10.21

在 X 上打开原推

The model that just beat GPT-6 Sol in cybersecurity now runs on two desk-sized boxes. At 2.9 bits. @MiaAI_lab shipped a new recipe of DeepSeek's V4.1 Flash 1017 for the DGX Spark pair, and I ran the full grid for a night so you know what it holds. The headline: a 1,039,888-token prompt answered, three needles planted across the span, all three back in order, eleven minutes end to end. Full agent grid run twice, at a 524,288 window and at a 1,048,576 window. THE BUILD Mia shipped a recipe for the 1017 checkpoint of DeepSeek's V4.1 Flash: a 2.9 bpw EXL3 quant, experimental, EAQ v1, served on @ashxhart TensorFold's native Zig engine instead of vLLM, tensor parallel across both Sparks. One script drives weights, engine, boot and smoke test. New checkpoint, new engine, new quant, stacked, which is exactly how failures hide. So I gridded it end to end. WHAT EAQ-EXL3 IS EXL3 is roughly 3-bit weights plus codebooks, decompressed layer by layer, which is how a model this size fits in two Sparks at all. EAQ v1 is not a new format. It is a better way of filling the old one. The pack is plain EXL3, and its per-tensor bit widths match the standard 2.9 bpw build across all 46,702 tensors. What changed is the conversion. Her card measures the result against the original weights: 0.0017 nats closer in paired KL than the standard build at the same widths, top-1 agreement 96.6 percent, the original's most likely token in her top-5 on 99.98 percent of positions. Same bytes, less distortion. THE NUMBERS Single-stream first, because it is the fair one. Prose 73.5 tok/s against her published 74.9, code 110.6 against 109.9, both inside two percent. Then her own bench script as shipped. One stream: 58.2, middle of her 55 to 61 band. Four streams: 119.2, two points above her 113 to 117. My code lane read 31 percent above her figure at two streams, and the advantage evaporated under her her testing method. THE WINDOW IS REAL A big advertised window is usually a banner. This one is not. The 1M window loads in 27 seconds with 7 to 9 GiB still free per node, and a prompt within 8,688 tokens of the ceiling is accepted and answered, needles 3/3. Prefill slides from 2,037 tok/s at 8k to 1,590 at 882k. Time to first token grows from 4 seconds to 555, an arithmetic no engine escapes. At a million tokens you are mostly paying to read. Decode does not degrade with history, 68 to 70 tok/s from 8k to 900k, and the 900k rung read slightly above the 524k one, which is noise, not a trend. CONCURRENCY: IT BATCHES, THEN IT QUEUES Throughput climbs to 119 tok/s wall aggregate at exactly four streams, the engine's batch size, and the best energy point too at 1.00 joules per token. Past four it declines to about 69 by 128 streams, yet every stream completes. 128 out of 128. Extra streams past the batch size buy queue position, not throughput. Garble checks: zero across 60 streams. TWO CAVEATS BEFORE YOU RUSH First, the engine's boot line says four streams at once, 0.00 GiB each, at the 1M window, 0.0 GiB left after a 12.2 GiB reserve. There is no static KV budget per stream. The cache grows as it fills, which is why one long prompt works and four concurrent long prompts are untested and unlikely to fit. Treat the 1M window as one long conversation, not four. Second, the quant. The card carries fidelity measurements against the original weights: teacher-forced, 100 conversations of at most 4,029 tokens, KL 0.0137 nats, top-1 agreement 96.6 percent. What it does not carry is long context, perplexity, or eval scores. GOTCHAS ON THE EARLY RECIPE Four hours went into things the README cannot tell you, because the repos moved after it was written. The pinned checkpoint commit was force-squashed off the Hub and answers "Invalid rev id." The pin never contained the tokenizer the prepare step needs, so it dies after 400 Gi (2/2)B of downloads. The fix is pulling the tokenizer from the base model as a real file, not a symlink. NFS needs no_root_squash, and the hf CLI must be on the non-interactive PATH. I have put the findings to her repo, if this is useful to others. THE VERDICT Extremely promising, Deepseek V4.1 Flash is already the world's most loved open source model, having this run on 256gb of unified memory on two DGX Spark, with 100 tok/s on code is a fantastic achievement by the local AI community. If the quality holds with real-use, then a ChatGPT 6 Sol class is now runnable on your desk, is it the "absolute frontier", no but thats insane on two boxes the size of your hands. Sources, findings, gotchas in reply 👇