Grok Bot 与 Muse 让 HBM 需求判断从看空转中性偏多
作者原本看空 HBM 容量，但 Grok Bot、Muse 这类长轨迹常驻 Agent 让 KV 缓存常驻、并发爆炸，压缩省下的容量被 Jevons 效应吃掉。
For a while I have been bearish on HBM capacity and bullish BW. Compression made capacity cheap and when v4.1 Flash took global KV down to 890B/tok and shoved the persistent cache into host DRAM and SSD I made my point again. BW is stacks x bus x width x pin rate. Height is NOT in that equation and I think base die is actually very important for BW. I still believe most of that. Everything I wrote assumed monolithic serving. You prompt, get an answer and then close out. KV lifecycle is short. Compress it 4x and you really do free up the cube buuuutttttttt agents don't close the tab how us humans would do. With the release of Grok Bot and Muse we know that the future is serving agents and now that they are more accessible to everyone we can only assume somewhat exponential growth. Context only grows with every took call, every page it reads, every screenshot, every handoff between Bots get appended. Agent trajectories are very long. Then we have to think about constant resumes. Tool returns in 2 seconds and the agent wants its cache back THIS INSTANT. And concurrency explodes like I mentioned earlier. A Grok Bot consumer is running a TEAM of bots around the clock all talking to each other. That is literally Jevons. Compression made context cheap per token so everyone burns way more tokens of context. Deepseek cut global KV bytes/tok by ~4x. Agents can EASILY multiply resident tokens by more than that. I'm not completely flipping though because BW is still king in my opinion. So I was bearish capacity but now I am neutral to slightly bullish capacity. I'll continue to expand on this