Jev as a research agent judge: steadier and cheaper than an LLM judge
A You.com agent monitoring business risks showed Jev matching an LLM judge on report quality, while the LLM scored the same threat 0.35, 0.68 and 0.50 across runs and missed 5 of 11 investigations that Jev caught, at 250x lower cost and 3-6x faster.
Jev is a huge power-up for research agents! @edwardirby built a @youdotcom agent that monitors business risks, and compared Jev to an ordinary LLM judge. • Same report quality • But the LLM flip-flopped: 0.35 -> 0.68 -> 0.50 on the same threat • the LLM missed 5 of 11 investigations. Jev missed none. • Jev was 250x cheaper, 3-6x faster Write-up 👇
This post quotes another post: “Building a risk agent without a frontier model using Jev for typed judgments”