Why Hermes Agent keeps rejecting Jev-style compaction
Teknium breaks down "Jev compaction": it only removes tool-call tokens, shrinking by half each round until nothing is left to compact, which is why Hermes Agent has rejected the PR every time.
Here's how the compaction in the tweet i'm quoting works: You get to 500K context, it does "jev compaction" -> Removes 250K of tokens of tool calls Then you're only compacted to 250K, not 50k which a summary would get you. Then you get back to 500K, it does jev compaction -> Can only remove 125K, because thats how many tokens of tool calls were left Now you start at 375K context, and you get back to 500K -> Can only remove 75K tokens, because thats what was left of the space for it to be tool calls Until there is no room left for "compacting" anything, this form of compaction has been PR'ed to Hermes Agent, many times, programmatically instead of with Jev though. It's been rejected every time. Because it is not a compaction method that can keep your session going forever, and costs more than summarizing the full context into a small amount, once.
Okay, everyone wants us to give the unbiased facts. Jev compaction here has a lot of problems. The biggest problems: - When I ran it on our public compaction eval (that you can run too), it resulted in a programmatic rule, that you dont need any Jev or other model for - it simply removed all tool calls from the chat history - This means you could do it for free, first of all, but second of all, it means you run into a viscious cycle. Every compaction, there are less and less tool calls to remove from the chat history, meaning it compacts less and less tokens, until there is no room for compaction, and you hit a hard stop, and can't compact anymore. - And, if you are cycling like this, every compaction breaks your cache, so you're paying 10x the price on input tokens, and keeping more input tokens each round means a higher baseline cost after compaction as well. Here's the full reproducible eval that you can run against Hermes Agent between the two (Hermes' vs Jev's): https://github.com/NousResearch/hermes-agent/pull/116246
This post quotes another post: x.com/tamarajtran/status/2100694549362553153
