Running DeepSeek V4.1 Flash on two RTX A6000s with expert offloading

Jeremy Tregunna @noozieOriginally posted 2026.10.28

Open original on X

I used to run llama 3 70B on my 2 RTX A6000 gpus. These days, i was wondering how to run deepseek v4.1 flash 1017, until i stumbled across project maya, gave it a spin, and on those same gpus and that same system from back then, got it to run by offloading experts and ngrams to main system memory. KV cache, active experts, attention, etc., run on the gpus. 96GB vram and 103.6GB of system memory.