jevPOST
Overnight Jev benchmark: cost, latency and injection blocking
Over 500 API calls cost just $0.03 and Jev caught a fake-approval injection on a self-trained agent kernel, but it still reported 1.00 confidence on a wrong answer.
Observer
@0bserverArchived 2026.09.18Originally posted 2026.09.17
Observer @0bserverOriginally posted 2026.09.17
Spent the night benchmarking Jev from @typesafeai. Ups: 20 extra questions cost +47ms; 500+ api calls, barely spent 0.03 $ It caught a fake-approval injection on my own fine-tuned agent kernel. I'd say %98 success rate (spent so much time with jev lol) Downs: it reported confidence 1.00 on a wrong answer, so no, you can't gate on it. YET. However, I'm impressed for a v1! Congrats @CompleteSkeptic and the team
x.com/0bserver/status/2100395569889906793Archived 2026.09.18 10:16