deepseek-v4.1IMAGE

DeepSeek V4.1 Flash on a 12GB GPU: 105–117 tok/s on an RTX 5070

GSQ-RCO quantization with the Strata inference engine runs DeepSeek-V4.1-Flash on 12GB consumer GPUs, hitting 105–117 tok/s on an RTX 5070 with 64GB RAM, with prefill notably faster than decode.

Md Ismail Šojal 🕷️

@0x0SojalSecArchived 6 days agoOriginally posted 2026.10.25

DeepSeek V4.1 Flash on a 12GB GPU: 105–117 tok/s on an RTX 5070 (image 1 of 2)

DeepSeek V4.1 Flash on a 12GB GPU: 105–117 tok/s on an RTX 5070 (image 2 of 2)

Md Ismail Šojal 🕷️ @0x0SojalSecOriginally posted 2026.10.25

Open original on X

You can now run a DeepSeek-V4.1-Flash locally 12GB GPU and Offloads consumer hardware. - GSQ-RCO quantizations with Strata inference engine. - RTX 5070 with 64GB/ 105-117 tok/s - Prefill is much faster than decode not bad - More RAM to faster decode - engine uses layer-major chunked reading, - for fun is a great.

x.com/0x0sojalsec/status/2114179761874612255Archived 2026.10.25 11:59