Jev-style scoring vs LLM decoding vs structured output for fixed-choice decisions

Daily Dose of Data Science @DailyDoseOfDS_Originally posted 2026.10.27

Open original on X

Jev-style scoring vs. LLM decoding vs. structured output, clearly explained: Many LLM requests do not need newly written text. Routing, classification, policy checks, and ranking usually already have a known set of valid answers, and the only question is which answer best matches the input. Consider a support ticket: "I was charged twice for the same subscription." The application must route it to either billing, technical support, or account access. With normal LLM decoding, the model goes through a full reasoning process. A decoding rule selects one token, appends it to the sequence, and runs the model again. This continues until the model responds with something like: "Based on the customer's query, this ticket should go to billing." The application then has to parse that text to recover the decision it needed from the beginning. Structured output handles this better, but the underlying process is still generation. A grammar or schema masks illegal next tokens at every step. The model cannot produce arbitrary prose, but it still generates/decodes the full JSON token by token: """ { "team": "billing" } """ This is useful when the application needs unknown values, nested fields, or tool arguments. The schema guarantees the shape of the response. Jev-style scoring handles fixed-choice decisions differently. The application provides the query, the input state, and the allowed answers before inference. This time, instead of decoding a sentence or JSON object, the model scores the predefined candidates: """ billing → 0.90 technical support → 0.10 account access → 0.00 """ The application receives the selected label and its probability distribution directly. The applicability entirely depends on the downstream use case. They expose different output contracts and perform different types of inference work. - LLM decoding is ideal when the application needs newly written text. It can also return a label, but it must generate and parse that label like any other response. - Structured output decoding is ideal when the values are not known beforehand, but the application requires a predictable schema. The model still generates the response token by token while the decoder prevents invalid structures. - Jev-style scoring is ideal when the valid answers are already known. It cannot write an explanation or produce an unknown field value. Instead, it compares the supplied candidates and returns their probabilities without decoding a sentence or JSON object. If you want to dive deeper into Jev, we also wrote a hands-on guide to building this Jev-style decision path with open models, entirely locally. Read it below.

This post quotes another post: x.com/1175166450832687104/status/210156361064449…