本地部署POST

llama.cpp adds DeepSeek V4.1 Flash, but smallest GGUF is 246 GiB

Upstream llama.cpp b11870 supports DeepSeek V4.1 Flash, but the author found the smallest working community GGUF (Q2_K) needs 246.3 GiB, far beyond 128 GB, and warns some fork GGUFs aren't interchangeable.

Wayne Lowry

@wikiwayneArchived 12 hours agoOriginally posted 2026.10.27

Wayne Lowry @wikiwayneOriginally posted 2026.10.27

Open original on X

DeepSeek V4.1 Flash support landed in upstream llama.cpp (b11870). Bad news for my GX10: the smallest working community GGUF I found (Q2_K) is 246.3 GiB. I have 128 GB. Math says no. 552B MoE, 8B active on prefill, 16B on decode. Gotcha: some fork GGUFs aren't interchangeable.

x.com/wikiwayne/status/2115005581702402202Archived 2026.10.28 12:04