WHAT NEW IN X11.03 TUE

AI collections from X · 22 collections · 65,263 posts

jevPOST

openJev Verdict reaches 74.9 on JevBench after fixing three inference bugs

The author ran openJev Verdict on a 151M base model without retraining and, after fixing three deterministic inference bugs, reached 74.9 on JevBench, explaining why Jev is replacing chat models for routing.

Hemant
@heman10xArchived 2026.09.22Originally posted 2026.09.22
Hemant @heman10xOriginally posted 2026.09.22
Open original on X

People asking why Jev is blowing up when it is "just a transformer encoder" are missing what developers dealt with for the last three years. Teams spent dollars calling 70B causal decoders for simple routing. You waited two seconds for tokens to stream, hoped the JSON parser would not crash on a markdown block, and dealt with prompt order bias where permuting the choices flipped the decision up to 27% of the time. A bidirectional encoder evaluates all candidates simultaneously in one forward pass. Flip rates stay below 5%, inference takes 25ms on local CPU, and serialization delays disappear. TypeSafe AI took a bidirectional encoder, calibrated its output probabilities, and exposed it through typed primitives (Choice, Score, Noul). I tested how far that architecture goes with openJev Verdict, an open-source version of TypeSafe AI's Jev on a 151M base model. Without retraining, fixing three deterministic inference bugs brought it to 74.9 on JevBench (#2 globally, trailing closed Jev by 0.5 points and outscoring 4B decoders). The reaction is not about architectural novelty. It is relief that teams can finally stop using chat models as glorified if-else statements.

x.com/heman10x/status/2102081143659167780Archived 2026.09.22 10:20