GB300 + 4×RTX PRO 6000 跑 V4.1 Flash：sidecar 方案吞吐估算

Stephen J Ronan MD @SJRonanMD原推发布于 2026.10.18

在 X 上打开原推

Nice apples-to-apples work from al-engr comparing a GB300 Station with 4x RTX PRO 6000 on DeepSeek V4.1 Flash. That Station keeps the cold experts in Grace memory. I run a second layout: the GB300 holds the hot experts and the Station's own RTX PRO 6000 serves the cold ones (sidecar). Our estimate for the sidecar on the same test, extrapolated from our Oct 6–7 runs and not yet measured on their grid: 1 user: ~380–470 tok/s vs 196–232 4 users: ~1,000–1,250 vs 506–586 8 users: ~1,300–1,600 vs 600–748 Prefill: ~40–50K tok/s vs ~7K Time to first token at 131K: ~3 s vs 18.6 s