对 Jev「无幻觉」宣传的质疑与基准拆解
作者认可 Jev 的速度成本，但指出它只是受限于预设标签的分类器，仍会选错，且 RLCD 方法与基准未公开。
Jev is definitely interesting. But there are still a few things I’d want to see before buying the bigger claims around it. TypeSafe loudly calls Jev “hallucination-free.” That’s mostly marketing. A model that isn’t generating free text simply cannot be compared with one that can. Jev is only allowed to choose from answers you define in advance. It can’t invent an invalid output - but it can still confidently choose the wrong one. TypeSafe’s RLCD method hasn’t been published, and neither have the calibration curves. That’s what I’d really like to see. The speed/cost numbers are impressive too, but notice the 193.6× faster / 444.6× cheaper headline is the best case - not the average. The early benchmark is interesting: Across 711 cases, Jev scored 67.8% - roughly in line with GPT-5.6 Terra at 67.9%, and below Sol and Opus 5 at 74.1% and 73.1%. At ~$0.0004 per case and ~0.4s, that’s impressive. But the benchmark itself still isn’t public. It covers just four TypeSafe-designed workflows, and I’d especially like to see how the reference answers were established - ideally against human judgment, not just model agreement. Strip away the branding and Jev currently looks like something simpler: A large pretrained transformer, post-trained into an extremely fast probabilistic classifier over caller-defined labels. That could be very useful. Whether it’s a new class of model is a much bigger claim.