DeepSeek V4.1 Flash on 4×DGX Spark: 155 tok/s code at 1M context
A user runs EXL3 2.9bpw DeepSeek V4.1 Flash on four DGX Sparks via TensorFold TP4, hitting ~155 tok/s code and ~87 prose at 1M context with 16 streams, and lists four setup gotchas.
Excellent - thank you! By using your recipe adjusted to @softpoo weights I managed to reach abliterated ~155 tok/s code / ~87 prose @ 1M context/16 streams! Working *really* well right now in smoke tests (will test more/deeply/heavily during the weekend). Summary if anyone wants to reproduce (give this to your favourite clanker): Setup bertholomus TensorFold TP4 — 4× DGX Spark (GB10) serving an EXL3 2.9 bpw abliterated DeepSeek-V4.1-Flash-1017. Expect 1 M context, 16 streams, ~155 tok/s code / ~87 prose (vs 151/72.5 for the neko engine on the same weights). What you need • Recipe: https://github.com/bertholomus/deepseek-v4.1-tensorfold-tp4-4xgb10 • Engine fork, branch deepseek-v41-zig-tp4: https://github.com/bertholomus/TensorFold • Weights: https://huggingface.co/neko-legends/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw-UNCENSORED-4xSpark • Kernel kit: https://huggingface.co/bertholomus/DeepSeek-V4.1-Flash-TensorFold-TP4-4xGB10 (kit/) • Engram shards 47 + 48: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017 • Build toolchain: Zig 0.17.0 + http://nvcr.io/nvidia/pytorch:26.07-py3 (sm_121) Four gotchas 1. The engine hard-codes RoCE GID index 5; on this fabric the RoCEv2 GID is 3 (4–7 empty), so every rank dies with VerbsFailed until that index is made configurable. 2. Their weight-readiness script is a no-op for the neko pack — its tensors are already BF16/F16/F32. Only chat/chat_template.jinja needs adding. 3. No /health endpoint — watchdogs must probe /v1/models.