双 RTX A6000 跑 V4.1 Flash：专家卸载到内存

Jeremy Tregunna @noozie原推发布于 2026.10.28

在 X 上打开原推

I used to run llama 3 70B on my 2 RTX A6000 gpus. These days, i was wondering how to run deepseek v4.1 flash 1017, until i stumbled across project maya, gave it a spin, and on those same gpus and that same system from back then, got it to run by offloading experts and ngrams to main system memory. KV cache, active experts, attention, etc., run on the gpus. 96GB vram and 103.6GB of system memory.