Jev 决策层接入渗透测试 agent 的对照实验
同一目标跑两遍、只切换 Jev 决策层：发现数 18 比 16，9 条结论被重新校准，耗时更短、决策层成本不到 5 美元。
if you're building on Jev, this is the most useful paper I've read on it - someone wired Jev into a real pentest agent and ran the same target twice, with the decision layer on and off. no marketing, just numbers the target had 13 known vulnerabilities. the agent hacked with claude-opus-4-8, up to 15 sub-agents. the only thing that changed between runs was Jev. what a Jev user actually learns from it: > Jev's job is judging, not answering. without it, the agent revised its own confidence 0 times. with Jev, 9 findings got recalibrated - false certainty dropped, solid results kept. > being cautious made it more useful, not less. the Jev run reported 18 findings vs 16, and surfaced 9 things beyond the seeded set (2 of them real bugs it was never told to look for). > it's fast and basically free. the Jev run finished in 26m53s vs 32m12s, and the entire decision layer cost under $5 - on a model that's ~$0.042 per 1M input tokens. you can afford to second-guess every single step. > calibration is the metric that matters. the paper frames it as ECE: ~0.08 for a well-calibrated model, ~0.25 for a bad one, with a 0.8 confidence threshold deciding what ships. that's the number to watch in your own stack. > and they're honest: one run each, not a proof. which is exactly why it's worth reading - it shows the pattern, not a sales chart. takeaway: don't replace your model with Jev. put it between the model and the action, as the thing that decides what's real
这条是引用着另一条写的：「长文：如何搭好 Jev 驱动的 agent 技术栈」