Jev decision layer tested in a real pentest agent: 18 vs 16 findings
Running the same target twice with only the Jev decision layer toggled, the agent reported 18 findings versus 16, recalibrated 9 conclusions, finished faster, and kept decision-layer cost under $5.
if you're building on Jev, this is the most useful paper I've read on it - someone wired Jev into a real pentest agent and ran the same target twice, with the decision layer on and off. no marketing, just numbers the target had 13 known vulnerabilities. the agent hacked with claude-opus-4-8, up to 15 sub-agents. the only thing that changed between runs was Jev. what a Jev user actually learns from it: > Jev's job is judging, not answering. without it, the agent revised its own confidence 0 times. with Jev, 9 findings got recalibrated - false certainty dropped, solid results kept. > being cautious made it more useful, not less. the Jev run reported 18 findings vs 16, and surfaced 9 things beyond the seeded set (2 of them real bugs it was never told to look for). > it's fast and basically free. the Jev run finished in 26m53s vs 32m12s, and the entire decision layer cost under $5 - on a model that's ~$0.042 per 1M input tokens. you can afford to second-guess every single step. > calibration is the metric that matters. the paper frames it as ECE: ~0.08 for a well-calibrated model, ~0.25 for a bad one, with a 0.8 confidence threshold deciding what ships. that's the number to watch in your own stack. > and they're honest: one run each, not a proof. which is exactly why it's worth reading - it shows the pattern, not a sales chart. takeaway: don't replace your model with Jev. put it between the model and the action, as the thing that decides what's real
This post quotes another post: “Long read: how to build a Jev-powered agent stack”