DeepInfer open-source Q&A: 155T tokens/day at ~$38 per GPU-hour
A user reads DeepInfer's Q&A: 155 trillion tokens a day, about $38 per GPU-hour at half price, while noting it runs only V4.1 Flash, omits cache and load balancing, and uses DeepSeek's own benchmarks.
deepseek in april 2025: we're too small to maintain a big open source inference engine deepseek today: here's the whole engine, apache 2.0 18 months. that's the turnaround the number i can't stop looking at in the deepinfer q&a: 155 trillion tokens a day ~$38 per GPU-hour, at half price renting a b200 costs $4-7/h. "AI is unprofitable" takes are going to have a rough week open weights = you get the recipe open engine = you get the kitchen the model decides what AI can do. the engine decides what it costs caveat before the hype: deepinfer runs one model (v4.1 flash), the cache system, PD split and load balancer aren't in it, and every benchmark is deepseek's own recommended setup is 128 GPUs. i'm only 128 short