Jev 用并行决策取代自回归生成

Manthan Gupta @manthanguptaa原推发布于 2026.09.17

在 X 上打开原推

Jev is one of the more interesting model launches I have seen recently because it asks a very simple question: Why are we using autoregressive LLMs as insanely expensive if statements inside software? The easiest way to think about it is: LLM: text -> generate tokens sequentially -> text But, Jev: state -> evaluate predefined decisions in parallel -> probabilities / typed values Suppose your application has some state like a customer record + their latest message, and you want to know: { "is_churn_risk": bool, "intent": Choice["refund", "support", "upgrade"], "urgency": Score[1, 5], } A normal LLM still has to autoregressively generate something like: { "is_churn_risk": true, "intent": "refund", "urgency": 4 } Even with structured outputs or constrained decoding, it is still fundamentally generating tokens one after another. Jev is designed differently. Instead of generating the answer as text, it appears to encode the input and directly produce distributions for all of these variables in parallel: P(churn=true) = 0.91 P(intent=refund) = 0.73 P(intent=support)= 0.21 P(intent=upgrade)= 0.06 P(urgency=1..5) = [...] So there is no need to generate {, "intent", :, "refund" and all the other syntactic machinery we normally pay for when the actual application only needs a decision. This sounds a bit like classic classification models, but the interesting part is that traditional classifiers usually have fixed labels defined during training. Jev appears to support dynamically specified typed decisions at runtime while still avoiding autoregressive generation. That explains where the speed claims could come from. If you ask an LLM to answer 30 classification-style questions, the execution looks roughly like: prefill -> token 1 -> token 2 -> token 3 -> ... -> token 300 Hundreds of sequential decode steps. Jev can instead do something much closer to: input -> forward pass -> 30 distributions in parallel Much more GPU-friendly. That's why their reported 70-500 ms latency and very large speed/cost improvements don’t sound architecturally crazy to me, although their biggest benchmark numbers are obviously on workloads that are shaped exactly like this kind of System-One decision problem. The second interesting part is RLCD: Reinforcement Learning for Calibrated Decisions. Instead of only optimizing for: "Did the model choose the right answer?" you also care about: "Does the probability it outputs actually mean something?" If Jev says: confidence = 0.90 then ideally, across decisions where it reports ~90% confidence, it should actually be correct roughly 90% of the time. That is extremely useful for real software: if confidence > 0.98: auto_execute() elif confidence > 0.70: ask_for_confirmation() else: escalate_to_human() That is much more useful than an LLM simply saying "I am highly confident." One claim I would be careful with is "can’t hallucinate." What Jev can strongly constrain is the output space. If your allowed choices are: refund support upgrade it can’t suddenly invent: banana But it can absolutely still choose the wrong valid option. So the way I would describe Jev is: It trades generative freedom for bounded outputs, parallelism, and calibrated uncertainty. And that is what makes it interesting. A huge amount of production AI isn’t actually trying to generate prose. It is trying to answer things like: Is this fraud? Which route should I take? Should I escalate? What category is this? Is this user likely to churn? Which action should execute next? For those workloads, forcing everything through next-token generation may be the wrong abstraction entirely.

这条是引用着另一条写的：「ChatGPT 联合发明人发布新模型 Jev」