How DeepSeek V4.1 Flash's 763B weights run on consumer GPUs

Yume_X @yume_arasakiOriginally posted 2026.10.23

Open original on X

When Deepseek V4.1 Flash came out, I lamented, this won't run on consumer hardware, then people made it happen. DeepSeek shipped the 1017 update. What's crazy is it beat GPT-6 Sol in cybersecurity. I mapped it all so you don't have to. THE MODEL Same 763B size 763B logical weights, 552B backbone MoE, 196B Engram, MIT, 1M context. The checkpoint is 510.3GB across 83 files: 289GB MXFP4 experts, 203GB Engram tables, the rest in attention and the draft head. Two facts decide every row. Only 6 of 384 experts fire per token, so expert weights shrink or stream. Engram is read on every token and punishes aggressive quantization. Every machine below picks expert bits and where Engram lives. HOW IT RUNS ON CARDS THAT CANNOT HOLD IT Decode activates only 16B of the 763B, six experts per layer out of 384 across 40 layers. None of these engines hold the whole model. They tier it: hottest experts in VRAM, next ~110GB in pinned system RAM the CPU computes or the GPU reads by DMA, coldest in the file on SSD, fetched a layer ahead of the router's picks. On two 3090s, three of four expert picks land on a GPU. On the 4090 build, 64GB of its 90GB RAM is an expert cache. Calibration ranks which experts run hot, and engines swap them as the conversation shifts: a coding session and a history chat end up with different experts on the card. Engram does ~48 small random reads per token, which is why NVMe is mandatory and why one build measured pulls 5x slower on exFAT. The 3060 build goes further: a 2-bit copy answers every word, only uncertain words get recomputed from exact FP4. THE DIAL NOBODY STATES Unsloth ships four Engram widths, MXFP8 at 202.8GB down to IQ4_NL at 110.6GB. Choose that before expert bits. On several boxes below it is the difference between fitting and not fitting. 1x DGX SPARK, 128GB A ~114GB VQ build holds 111 to 113GB resident, 203GB of Engram streaming from SSD: 39.4 tok/s weighted decode, ~1,145 prefill at 32K, on a 110K prompt. Flagged by the source as ~1.6 bits, not full-checkpoint quality. A week-older result on the previous checkpoint, better labelled: Cruz measured 24.7 tok/s warm-repeat at 1.59-bit SAGE-EXL3, code top-1 97.9%, KLD 0.1885. The only single-Spark number that states what the quant costs. 2x DGX SPARK This tier is contested, six competing recipes. Ours is TensorFold with Mia's EAQ 2.9bpw at 1M context, four parallel streams. Measured this week: 142.97 tok/s structured, 124.35 code, 55.53 prose. Thinking off, median of three, clocks at 2,411 MHz. TensorFold beats the vLLM kit on identical boxes, same pair, weights, prompts, day: text 27.0 to 56.8, code 39.9 to 92.2, a 64K needle 80.4s to 35.9s. tonyd2wild's TP2 lane: 73.1 at one user, 135.8 at four, 1M-token needle found. HeNryous at 2.75bpw posts KLD 0.0201, top-1 95.96%, best published fidelity here. sudoingX reports 147.6 with no config or context, so that stays a report, not a measurement. 3x AND 4x DGX SPARK Official quant, no requantization. The TP4 line outperforms its own announcement: prose 87.7 at one stream, code 124.8, 342.7 aggregate at sixteen, cold prefill 4.1k to 5.9k, a 1,011,084-token needle passing. TP3 gives 51 prose, ~2,000 prefill. A China OEM four-box ring, no switch because one costs about a fifth of a Spark, runs SGLang TP4 on official weights: code 90 single, 296 at eight, prefill 2.9k at 1M. 8x DGX SPARK Eight GB10 on 200G RoCE: 79 to 83 coding single-stream, 241 at C6, an 8.1M-token KV pool, needle passing at 900K. The labelling problem: it serves dealignai's uncensored FP8 build, not stock 1017. Not official-checkpoint numbers. 5x CMP 170HX, 64GB The best engineering result of the week, about layout rather than hardware. TP4 across four of five cards: 216.7 structured, 183.1 code, 320.6 aggregate at eight users. PP5 delivers 3,044 prefill at 255K but single-stream decode collapses to 1.0 to 13.9 tok/s, because PCIe Gen2 x4 serialises the pipeline. Against 183.1, those five cards land 13x to 180x apart. Layout beats node count. RTX PRO 6000 0xSero went 410 to 609 to 675.9 tok/s in three days on four cards. The screenshot settles the last one: original weights, not a quant, C1, 528K context. A separate logged rig, mislabeled "6 x RTX 4090" in its own header but actually TP4 on four PRO 6000s: 115.7 single-stream, 901 peak at C16, 7,643 prefill at 32K, on a 9.7M-token KV cache. The only run publishing its full serving command and image integrity manifest. THE CONSUMER ROWS 2x RTX 3090 plus 128GB DDR4: 28.9 tok/s prose, 470 prefill, exact routing, no pruning. The price is quality, KL 0.373, top-1 74.4%. 1x RTX 5090: 12.0 prose, 15.2 on a 3K agent prompt. Do not carry over the 21 tok/s from the older checkpoint. Different build, and here ~40GB of experts on disk set the pace. 1x RTX 4090 24GB, 90GB RAM: 26.8 tok/s, 180 to 320 prefill, 512K context, HumanEval 21/21, tool calls 10/10, agentic loops 3/3. The feasibility checkers still say this card cannot run it, because math from model-file size says no. The measured row beats the estimate: a 152 GiB tiered quant, 90GB of host RAM. The gate was never VRAM. It was RAM and disk. RTX 3060 12GB with 64GB: 5.8 cold, 7.4 warm code tok/s. Runs, not a workstation. A Ryzen AI Max+ 395 mini PC with an R9700 in a USB4 dock, on Windows: 14 tok/s, 128K context, 1-bit. MAC M3 Ultra 512GB, native MXFP4 GGUF: 37.6 tok/s at 8K, Engram on SSD. The 432GB MLX build needs ~476GB and only fits by raising the wired memory limit. 128GB Macs are out. WHAT THIS SAYS Ten days ago this model mapped across nine tiers. Four days later, six had moved. The 5090 got slower once measured properly. The 4090 went from "cannot run it" to 26.8 tok/s. A 3060 runs it. Layout can cost 180x on the same silicon. The ladder is no longer GPU count. It is host RAM, disk speed, Engram placement, engine fit. Sources in reply 👇

This post quotes another post: “DeepSeek V4.1 Flash at 2.9bpw on dual DGX Spark: 73.5 tok/s, 3/3 needles at 1M tokens”