Jev 路由实测:错载工具从 16.8% 降到 7.3%,但弄坏 7 个请求
488 条请求、182 个技能的 Hermes 测试:Jev 建议降低错载率,也修好 37 条、弄坏 7 条原本正确的请求。
ONE JEV SUGGESTION CUT WRONG TOOL LOADS FROM 16.8% TO 7.3% THEN BROKE 7 REQUESTS THE AGENT ALREADY GOT RIGHT That is exactly the kind of benchmark AI launches usually hide TypeSafe tested 488 requests against a Hermes roster containing 182 skills Without Jev, Claude Haiku loaded the wrong skill on 16.8% of covered requests and loaded a skill unnecessarily on 9.8% of uncovered requests With a Jev suggestion, those rates fell to 7.3% and 4.0% But the router was not free intelligence It fixed 37 covered requests and damaged seven requests the agent had previously handled correctly That is the real agent-routing problem. A cheap model can shrink the tool space before an expensive model thinks. It can also confidently remove the tool that would have solved the task So routing needs a real NONE option, measured thresholds and logs showing when the suggestion changed the final choice Cheaper decisions help Unmeasured decisions just move the failure upstream
这条是引用着另一条写的:x.com/fluixoo/status/2103123493567025321