763B DeepSeek V4.1 Flash runs on 12GB VRAM via GSQ-RCO quantization
IST Austria's GSQ-RCO quantization plus the Strata engine runs the 763B V4.1 Flash on consumer GPUs: ~5 tok/s on RTX 5070 and 8–10 tok/s on RTX 3060, but it needs 239GB of storage and plenty of RAM.
DeepSeek-V4.1-Flash, with 763 billion parameters, can now run locally on GPUs with just 12 GB of VRAM. Researchers at IST Austria quantized the model using GSQ-RCO, while the Strata inference engine made it work on consumer GPUs. Benchmarks: • RTX 5070 + 64 GB RAM: ~5 tokens/s generation, up to 231 tokens/s prompt processing. • RTX 3060 + 46 GB RAM: 8–10 tokens/s generation, ~149 tokens/s prompt processing. The catch: at least 239 GB of free storage and plenty of RAM. It's slow for complex agentic tasks, but running a 763B model on a regular PC is a major step forward in quantization and inference optimization. https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF