Jev beats Claude and rules on fraud ring classification test
Tested on a real fraud ring, Jev reached 93% recall, ahead of the 90% velocity rules written by a data analyst agent and the 62% of an LLM making the classification decision.
Jev vs. Claude vs. Rules: We ran a real fraud ring through all three. The headline result is that Jev classified best at 93% recall, followed closely by the velocity rules our Data Analyst Agent proposed at 90%, with an LLM making the classification decision trailing at 62%. This shows Jev is great at reasoning and classification jobs compared to LLMs. The fact that Jev was so close to a Data Analyst agent (which could write multiple SQL queries to investigate the fraud ring) is the real finding. Classification in code was already near-optimal, and Jev matched it. Read more in the article: https://x.com/soupsranjan/status/2101711928053920067
This post quotes another post: x.com/soupsranjan/status/2101711928053920067