WHAT NEW IN X11.02 MON

X 上的 AI 合集档案 · 22 个合集 · 64,204 条

jevPOST

路由实验质疑 Jev 到底加了什么

13 模型 5835 条查询的独立路由实验里,Jev+检索 62.4%,去掉 Jev 检索单独也是 62.4%,作者质疑 Jev 在路由决策中究竟贡献了什么。

Sinda | SKatalyst AI
@SKatalystAI2026.09.22收录原推发布于 2026.09.21
Sinda | SKatalyst AI @SKatalystAI原推发布于 2026.09.21
在 X 上打开原推

Found something about Jev that surprised me: in an independent routing experiment across 13 models and 5,835 held-out queries, Jev + retrieval reached 62.4% accuracy, beating the best single model at 60.3%. But when they removed Jev, retrieval alone still scored 62.4%. The routing gain wasn’t coming from predicting whether a task was easy or hard. It came from knowing which models had historically performed well on similar tasks. Even crazier, an oracle choosing the cheapest model that could correctly answer each query would reach 82.6% accuracy at $4.83 per 1,000 queries, compared with 62.4% at $26.73 for the tested router. So if removing Jev produced the exact same 62.4%, what exactly is Jev adding to the routing decision?

x.com/skatalystai/status/2101946640005275909采集于 2026.09.22 10:17