DeepSeek 首次公开上下文缓存规模：85% 命中率省下约 2 万张 GPU
V4.1 Flash 每 token KV 仅 890 字节，是 V4 Flash 的四分之一；85% 命中率下无缓存需多 3.4–4.1 倍 GPU，100T token 上下文仅占 89PB。
At an 85% hit rate, only about 1/7 input token needs prefill. Without the cache, DS would need 3.4 to 4.1 times as many GPUs for the same traffic, about 20k more. The Q&A's storage server has 20 SSDs of 30.73 TB each, so the whole cache fits on about 183 servers. The GPU number is on the low side, because recomputing a long context costs more per token than recomputing a short one. The storage is cheap because the model's KV is small. V4.1 Flash uses 890 bytes per token, a quarter of what V4 Flash used. With FP8 KV, DeepSeek-V3 uses about 35 KB per token, and Llama 3.1 70B about 160 KB. At 890 bytes, 100T tokens of context take 89 PB. At V3's size, they would take 3.5 EB. DeepSeek has cached context on disk since August 2024. This is the first time they released how big the cache is. The cache system is also one of the parts DS did not disclose. The small KV is part of the model itself, and the weights are open under MIT.