Decision model benchmark for gating security scans: Jev 1.14 and Decider lead
Corridor uses Jev to gate when security scans run and benchmarks it against decision models from OpenAI, Microsoft, Perplexity and others, with Jev 1.14 and Decider performing best at every budget.
We recently demonstrated that TypeSafe’s Jev could be used to gate when our security scan should be run. Since then, OpenAI, Microsoft, Perplexity, and a dozen others shipped decision models of their own. So we ran all of them through the same benchmark for this task. The top two models are Perplexity's Decider and TypeSafe's Jev 1.14. At any budget for latency, cost, or precision, Decider or Jev catches the most vulnerable posts. The rest are at or below the performance of an LLM gate. Full results and the dataset in our latest post: https://www.corridor.dev/blog/decision-models-benchmark