jeval：给 Jev 置信度阈值做校准与 CI 回归
作者开源 jeval，用你的日志验证 Jev 输出的 0.8 置信度实际命中率，并按误判成本算出该设的阈值，模型更新变差时 CI 会报警。
Thinking about using Jev? Jev doesn't write text. It just makes judgments — "this ticket goes to billing" — and hands you a number for how sure it is. Fast and cheap. But as an engineer, the thing that actually matters isn't model performance. It's whether this fits into your service properly. Because eventually you have to write this line: if confidence > 0.8: auto_handle() Where did 0.8 come from? Usually copied from a docs example. Nobody ever measured what 0.8 actually means on your data. Too low and things break. Too high and you've automated nothing. And you can't tell which way you're wrong. That's why I'm building jeval. Feed it your logs and it tells you "0.8 was actually 77%." Give it your cost of a wrong call, and it computes where the line should be. When a model update quietly makes things worse, your CI catches it. Not Jev-only. Works with any classifier that returns confidence. GitHub: https://github.com/rlaope/jeval #jev #ai #opensource