deepseek-v4.1IMAGE
552B DeepSeek V4.1 Flash on two RTX 3090s: 34.9 tok/s decode
lxe's MIT-licensed dsx keeps hot experts in VRAM and cold T16 experts in RAM, hitting 34.9 tok/s decode on two RTX 3090s plus 128GB RAM, with CUDA 13 and ~305GB NVMe required.
Sympathy
@SympathyLabsArchived 9 days agoOriginally posted 2026.10.23
Sympathy @SympathyLabsOriginally posted 2026.10.23
running 552b deepseek-v4.1-flash-1017 on two rtx 3090s + 128gb ram? @lxe's dsx keeps hot experts in vram and cold t16 experts in ram for 34.9 tok/s decode (36.8 with --defer-rank 3). mit-licensed; needs cuda 13 and ~305gb nvme. https://github.com/lxe/dsx
x.com/sympathylabs/status/2113377753286468048Archived 2026.10.23 12:08