Seven overlooked implementation details in Jev's docs
A walkthrough of Jev's parallel sampler, the caps on its Choice, Score and Noul question types, the shared 64K and 32K token budgets, the 12x savings from batch calls, and the design contract of giving candidates and letting it choose.
MOST PEOPLE TALKING ABOUT JEV HAVE NEVER READ PAST THE PRICING PAGE. Here's what's actually buried in the docs. 1/ Jev plays DOOM. Not as a gimmick. TypeSafe ran it live, roughly 10 decisions per second, structured game state in, action out. No frames. No pixels. Just typed state and a choice every 100ms. Cost to run it continuously: about $7 an hour. That's the real pitch. A model that reacts inside a loop where a 3 to 30 second LLM call isn't even an option. 2/ It's not autoregressive. Jev uses a parallel sampler that generates every output in a single pass, not token by token. That's the actual mechanism behind the 70 to 500ms latency, not just "smaller model, faster server." 3/ There are exactly three question types, and each one has a hard ceiling. Choice: max 255 options. Score: 2 to 10 levels, no more. Noul: a boolean, and the probability itself IS the confidence. There's no separate confidence field on a Noul question. 4/ You get a shared 64K token budget across state and questions combined. But there's a second, tighter constraint hiding under that one: your state plus your single longest question has to fit in about 32K tokens. Roughly 150,000 characters. Most people hit this limit before they hit the first one. 5/ Batching isn't a nice to have, it's the entire cost model. 13 questions in one call measured out at 12.2x cheaper and 10x faster than 13 separate calls, with no drop in answer quality from batching itself. If you're calling Jev once per question, you're paying a tax you don't have to. 6/ The single biggest mistake people make writing questions for it: asking Jev to extract or generate something. Don't ask it to name a card. Give it the deck and ask it to pick one. Candidates in, choice out. That's the entire design contract. 7/ Jev cannot count, do date math, or compare versions reliably. Not a bug. A boundary. Anything that's arithmetic gets computed in your code and handed to Jev as a pre-computed fact, not asked as a question. 8/ Accuracy quietly degrades as you cram unrelated content into the state. It's not context rot the way LLMs get it. It's state pollution. Every irrelevant token in state is competing for weight against the one thing you actually need judged. 9/ If you're on the Vercel AI SDK, confidence doesn't live where you'd expect. It's not inside the answers object. It's at result.providerMetadata.typesafe.confidence, keyed by question id. Miss that and you'll think confidence scoring doesn't exist. 10/ Output tokens are free. Not cheap, free. TypeSafe calls them "too cheap to meter" because the output is a typed answer, not generated text. At $0.042 per million input tokens, a 300 token classification costs about $0.0000126. That's the number that makes per record AI classification make sense for the first time. Most people are still evaluating Jev like it's a smaller, faster chatbot. It isn't one. It's a completely different primitive, and the docs make that obvious if you actually read them. Follow @cyrilXBT, I read the boring parts of the docs so you don't have to. https://x.com/TheMattBerman/status/2100654891756589230/video/1
This post quotes another post: x.com/1373665408193036293/status/210151513079829…