IST Austria's Strata quantization runs 763B V4.1 Flash on a 12GB GPU at 5–10 tok/s
IST Austria released a heavily quantized DeepSeek-V4.1-Flash that runs on a single 12GB GPU by offloading to RAM and SSD: about 5 tok/s on an RTX 5070 with 64GB RAM and 8–10 tok/s on an RTX 3060, needing 239GB+ of disk.
763B DEEPSEEK ON A 12GB GPU? HOLY SH*T researchers at IST Austria released a heavily quantized DeepSeek-V4.1-Flash. Strata can run it on a single 12GB GPU by keeping most of the model in RAM and reading the rest from an SSD ▸ RTX 5070 + 64GB RAM: about 5 output tokens/s ▸ RTX 3060 + 46GB RAM: about 8–10 output tokens/s in a separate test ▸ you still need 239GB of disk for the smallest package, or 257GB for the tested 2-bit version → “runs on 12GB VRAM” does not mean “fits in 12GB.” this is an impressive local run, but i wouldn't call it a fast setup for coding agents yet links in first reply ↓