决策模型与 LLM 的区别：Jev 的延迟、成本与校准

Aurimas Griciūnas @Aurimas_Gr原推发布于 2026.10.20

在 X 上打开原推

𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻 𝗺𝗼𝗱𝗲𝗹𝘀 (𝗝𝗲𝘃 𝗯𝗲𝗶𝗻𝗴 𝗼𝗻𝗲 𝗼𝗳 𝘁𝗵𝗲𝗺) 𝘀𝗵𝗶𝗽𝗽𝗲𝗱 𝗮𝘀 𝗮 𝗽𝗿𝗼𝗱𝘂𝗰𝘁 𝗰𝗮𝘁𝗲𝗴𝗼𝗿𝘆 𝗹𝗮𝘀𝘁 𝗺𝗼𝗻𝘁𝗵. Here is why they are different to LLMs even though both can achieve the same result 👇 In decision models, you pass state, text or JSON, plus questions whose answers you declare in code. You get back a probability over those answers. That’s it, and it is also the selling point, nothing to parse and validate. On the other hand, asking an LLM for a bounded answer means: 𝟭. Option list and rubric go in the prompt 𝟮. The model generates tokens, one forward pass each 𝟯. You parse the string, validate the schema, retry on invalidation 𝟰. The answer set is enforced after the text already exists A decision model on the other hand makes the answer space an input, bounded before inference runs. With one scoring pass, you get a probability for every option you declare. Basically, a typed value with confidence attached. Pros: ✅ You get a distribution, not a label. Per-option probability allows you to write the decision policy once: act above a threshold, cheap second check in the middle band, escalate the tail. ✅ Latency that significantly reduces the total latency of the loop. Jev runs P50 at 0.21 s and P95 at 0.34 s, and OpenAI advertises Decisions API to be close to 150 ms against roughly 1.6 s for the same model answering normally. ✅ No cost on the output side. Jev is $0.042 per million input tokens with free output. Generally, output is the expensive half of a judgment. What to be aware of: ❌ Calibrated is a vendor claim. Check it against your own labelled set calibrate thresholds in house. ❌ Accuracy degrades as irrelevant state grows. As always, keep your context as lean as possible. ❌ Option names are more important for Decision Heads than the rubrics explaining them. Make sure to name them accordingly. I.e. refund_full_no_return instead of option_a. 👉 Count the model calls in your agent loop, e.g.: ➡️ Routing ➡️ Is the retrieved context sufficient ➡️ Is the draft grounded ➡️ Do we escalate All of these have finite answer sets and run as full LLM calls with schema on top today. However, only one really needs to generate some text. ❗️ There is still a lot of use case for complex structure outputs, not every problem can be solved with simple classifiers. Looking forward to seeing decision models proliferate in architectures our clients are building! Which decisions in your system are a full LLM call that did not need to be? 👇