Eight V4.1 Flash agents on one 8x V100 server at 40-50 tok/s

A used 8x V100 server running a custom Rust + CUDA engine serves eight V4.1 Flash agents at about 40-50 tok/s, roughly $0.20-0.25 per million output tokens.

Aleksey Smolenchuk

@lxeArchived 9 days agoOriginally posted 2026.10.23

Aleksey Smolenchuk @lxeOriginally posted 2026.10.23

Open original on X

8 agents running DeepSeek-V4.1-Flash at once at ~40-50tps, All 8 are served from one used 8x V100 server running a custom a Rust + CUDA engine. This is about $0.20–0.25 per million output tokens on a spot-rented marketplace box.

x.com/lxe/status/2113382630284755403Archived 2026.10.23 12:06

Open original on X