Hermes 用可复现评测对比 Jev 压缩效果
Teknium 把 Jev 与 Hermes 现有压缩策略放进同一 benchmark：Jev 在 115K token 下召回 75.5%，Hermes 在 55K 下 78.9%，评测已并入 Hermes 供复现。
This is how agent features should be evaluated There was a claim that Jev could make compaction faster and better, so Teknium actually tested it against Hermes with a reproducible eval. The result: Hermes production compaction: 78.9% recall @ 55K tokens Jev: 75.5% recall @ 115K tokens And the interesting part is why. At its default threshold, Jev dropped every one of the 851 tool-call candidates in the test. At the same token budget, Jev’s ranking tied simply keeping the most recent results. Over repeated compactions, the amount it could remove kept shrinking because user and assistant text remained. So Hermes keeps its current summary + session_search approach. But they didn’t just say “ours is better.” They merged the eval into Hermes so anyone can reproduce the results and test future compaction strategies against the same benchmark.
这条是引用着另一条写的：x.com/teknium/status/2101398453578555898