用 agent 自身轨迹训练 Jev 评分器，防住刷分作弊

laxman @llmluthor原推发布于 2026.10.09

在 X 上打开原推

The trick: we trained our own Jevs on the agent’s own traces step Jev: grades every search the agent makes final Jev: reads the whole trace and predicts if the answer is right (AUC 0.92) We also tried a big hosted judge off the shelf. The agent hacked it in ~3 updates by not searching at all, just answering instantly. The judge rated those answers ~70% likely correct. All of them were wrong. 0% accuracy. It learned that if you take zero steps, there’s nothing to grade 😭