用 Jev 决策模型做 LLM Judge 评测
作者推荐用 System One 类决策模型（如 Jev）替代长文本 LLM Judge，并附两篇 Jev-as-a-Judge 实操资源与论文。
Learn to build evals, folks. Highly recommended reading on using decision models for evals. From our internal evals, System One models like Jev are a great fit for LLM Judges. If you want to go beyond Hamel's great write-up, I've also put together a few resources on applying decision models to LLM judges. (bookmark it) 1) Jev-as-a-Judge for Agent Evaluations - https://academy.dair.ai/dashboard/resources/jev-as-a-judge 2) Jev-as-a-Judge: Evaluating Agents with a Decision Model - https://academy.dair.ai/labs/jev-as-a-judge-for-agent-evals Bonus paper I would recommend, with a few other ideas that combine System One and System Two models: Jev-as-a-Judge paper - https://arxiv.org/abs/2609.26550
这条是引用着另一条写的：x.com/hamelhusain/status/2112588257104216294