Jev 在 44 个基准上做失败检测的实测总结
把 Jev 当 LLM 输出校验器，44 个基准、7193 条响应上测幻觉、提示注入与数据泄露，中位 AUROC 0.886，成本比 LLM 评审低约 63 倍。
Chinese students just found the best way to use JEV for any LLM or AI agent - released a PDF research the shift: I pasted it into Claude and GPT - and cut my costs by~63х here’s what they found across 44 benchmarks: 1 → 7,193 responses, 10 types of failure. Jev was tested on hallucinations, prompt injections, data leaks, and other AI failures 2 → One simple question worked: 0.886 median AUROC, beating trained baselines on 25 of 31 benchmarks without task-specific training 3 → Context beat clever prompting - give Jev the source or rule it needs to check the answer against 4 → Keep the probability, not just "yes" or "no" - Fitting a threshold on 10 labeled examples raised median F1 from 0.706 to 0.793 5 → Among the 50% most confident decisions, median accuracy reached 0.933 - send uncertain cases for another review 6 → Jev even helped uncover labeling errors in three benchmarks. Sometimes the test’s "correct answer" was the problem 7 → 11.4 questions per call, with 0.31-second median latency - on 19 benchmarks, checking cost $0.30 vs $18.96 with LLM judges - roughly 63× cheaper the result: It will made your setup CHEAPER and FASTER than what 95% of people are running Copy the Jev setup researchers tested across 44 benchmarks - then read the full Jev architecture ↓
这条是引用着另一条写的：x.com/polydao/status/2103689373774483815