用 Jev 置信度分数给 agent 流水线设门槛
一位开发者用 Jev 的置信度分数按风险分级设阈值(只读 0.5、退款 0.7、升级 0.8、破坏性操作 0.9),低于 0.5 直接转人工。
Jev can gate an entire agent pipeline with one number TypeSafe's decision model. One builder's setup: 0.91 vs 0.22 on the same 4 options, same top answer. One clears every rung, the other fails all four. > PAIR: every call returns the answer and a confidence score: how tightly the probability sits on one option > LADDER: the bar rises with the stakes: read-only 0.5, refund 0.7, escalation 0.8, destructive 0.9 > FLOOR: below 0.5 nothing fires, ticket goes straight to a human > TRAP: Noul has no confidence field. The probability is the signal, and 0.5 means the model can't tell yes from no > VERIFY: confident "done" doesn't prove the file was saved. Confidence is not permission > PIN: jev-latest pointed to jev-1.13.0 in September. Aliases move, thresholds tuned on one version break silently on the next Confidence measures concentration, not correctness. A 0.91 can still be dead wrong. Money and destructive calls need an outcome check after the gate. Set the rungs by cost of failure, tune on labeled data, pin the version. The terminal below walks the whole ladder ↓
这条是引用着另一条写的:x.com/ridark_eth/status/2112201965123809623