763B 的 V4.1 Flash 经 GSQ-RCO 量化在 12GB 显存本地跑起来
IST Austria 用 GSQ-RCO 量化、Strata 推理引擎让 763B 的 V4.1 Flash 跑在消费级显卡：RTX 5070 约 5 tok/s、RTX 3060 8–10 tok/s，但需 239GB 存储和大量内存。
DeepSeek-V4.1-Flash, with 763 billion parameters, can now run locally on GPUs with just 12 GB of VRAM. Researchers at IST Austria quantized the model using GSQ-RCO, while the Strata inference engine made it work on consumer GPUs. Benchmarks: • RTX 5070 + 64 GB RAM: ~5 tokens/s generation, up to 231 tokens/s prompt processing. • RTX 3060 + 46 GB RAM: 8–10 tokens/s generation, ~149 tokens/s prompt processing. The catch: at least 239 GB of free storage and plenty of RAM. It's slow for complex agentic tasks, but running a 763B model on a regular PC is a major step forward in quantization and inference optimization. https://huggingface.co/ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF