Scoring history with Jev and gating actions on confidence
The author uses Jev to score history entries into four tiers, cutting 1M tokens to 86K, routes low-confidence questions to Kimi K3, and keeps a fixed 0.95 publish threshold while recalibrating the 0.85 line weekly.
YOUR AI AGENT IS SCAMMING YOU. EVERY CALL IT RESENDS YOUR WHOLE HISTORY AND BILLS YOU FOR IT. 1,000,000 TOKENS → 86,000. IN ONE SECOND. Old logs. Dead ends. Every "ok." By hour three you're paying to resend a novel so the model can answer one question. The fix is a filter. JEV scores every history entry: critical, useful, background, junk. Critical and useful stay. The rest gets deleted before the big model reads anything. 1M tokens down to 86K. About a second. Second fix: stop paying the big model to be unsure. JEV answers the small questions with a confidence score. Above 0.85, code runs the action. Below it, Kimi K3 takes over. The trap: 0.85 doesn't mean 85% right. It's just a number. So once a week I check it against real results and move the line by 0.02. Third rule never moves. Nothing publishes under 0.95. Under that, it comes to me. Cut the history. Put the threshold in code. Pay the expensive model only when the cheap one isn't sure. Save this before your next API bill ↓
This post quotes another post: x.com/starmexxx/status/2108159970017488953