deepseek-v4.1POST
DeepSeek V4.1 Flash on vLLM: sleep scheduling on a GB300
Over 7.3 hours and 1,214 requests on a GB300, the model slept 51% of the time, background evals got 3.7 free GPU-hours, and 85% of requests waited nothing extra (1.3s average added wait).
Ben Buschmann
@benbuschmannArchived 1 day agoOriginally posted 2026.10.27
Ben Buschmann @benbuschmannOriginally posted 2026.10.27
One night on a GB300: - 7.3 h, 1,214 requests - model asleep 51% of the time - background eval got 3.7 GPU-hours - 85% of requests waited nothing extra - average added wait 1.3 s Model: DeepSeek V4.1 Flash on vLLM. Sleeps in ~10 s, wakes in ~2.5 s.
x.com/benbuschmann/status/2114750398829596698Archived 2026.10.27 12:09