双 3090 家用机上跑 V4.1 Flash 320B MoE：26–32 tok/s
作者分享双 RTX 3090 + 256GB DDR4 的家用配置，EXL3 ~3bpw 量化下 DeepSeek V4.1 Flash 解码 26–32 t/s、prefill 737–830 t/s，10 万 prompt 约 2 分钟。
I feel like we are pusing GLM 5.3 Flash and DeepSeek 4.1 Flash, both with EXL3 ~3bpw on dual 3090, and the speed numbers are close to it limit already. This is some shareing about my home LLM rig (spec + what each part does) - GPUs: 2x RTX 3090 24GB (used), 300W cap, holds the hottest experts and runs all the compute; more VRAM = faster decode because we can hold more hot experts. RTX 3090 also recently got a doping by unlocking hardware (hidden by Nvidia), now the prefill speed of 3090 is even better and feel so much more usable for both model. - Mainboard: ASUS Pro WS WRX80E-SAG, 8 memory channels feed the RAM expert tier; full x16 Gen4 for both cards; room for two more 3090. - RAM: 256GB DDR4 (8x 32GB), fits all ~197GB of experts, so nothing reads from disk during generation and I also actually have all 8 channel so less bottle neck when I need to read cold expert onto VRAM. - CPU: Threadripper PRO 3955WX, 16 cores, computes part of the experts in RAM in parallel with the GPUs. - Storage: 1TB NVMe, model files and the coldest tier - A tons of fans to heat off to rig, I even open the side panel to make it heat free, so my room is now always warm. Runs: DeepSeek-V4.1-Flash 320B MoE, 262k context Result: 26-32 t/s decode, 737-830 t/s prefill, 100k prompt in ~2 min. You can actually boots the prefill speed above 1000 by sacrify some decode speed, and I don't know how to get them both yet. All report and note can be found in my repo below. You need something simple and feed different model config, go for Maya repo.