Jev's System 1/System 2 split for agent decisions
The author argues that Jev itself may be forgotten, but its idea of not using LLMs for control-flow decisions and instead using a small model for a single forward-pass score will be adopted across the industry.
Jev from http://TypeSafe.ai is fantastic, but it will probably be forgotten in six weeks (like most AI breakthrough headlines). Yet the core idea which they introduced is going to be adopted by everyone Jev's idea is simple. Don't use LLMs for everything, use smaller model with very little output for decisions. Current approach which use LLM inside agents just to make control-flow decision "is this bash command destructive?" or "shall we ask confirmation from the user?" is an overkill. Even with JSON schema output mode, you're still waiting on a sequential token-by-token decode loop just to get back a boolean or an enum. Jev suggested division of labor between System 1 model and System 2 model. Generative LLMs for planning, writing code, and talking to users Fast, calibrated decision models for tool approvals, routing, and stuck-loop detection (which is what Jev is) My guess is that the industry will catch up to this idea very soon, and it will get implemented across the board First. Dedicated decision endpoints from major labs. OpenAI, Anthropic, and Google already use these internally as Process Reward Models (PRMs), safety classifiers, and verifiers. They'll inevitably expose sub-50ms endpoints that take your context and score a schema in a single forward pass without generating text Second. We’ll see tiny 0.5B local models for tool gating running directly in the harness to classify shell commands and flag risky actions or PII data with zero network latency Whether TypeSafe as a company survives doesn't really matter. Burning tokens just to make basic control-flow decisions is on its way out