Splitting an agent into LLM writes, JEV decides, code acts
The author lays out an architecture where the LLM only writes text, JEV returns a choice, score or probability with confidence, and code enforces hard rules, citing demos like 1,018 papers classified for 8 cents.
a founder told me his agent bill dropped 90% last month. and the agent got faster. “i was paying a frontier model to answer yes or no. thousands of times a day.” i asked what changed. he showed me this architecture, and the problem was obvious right away. most agents are one giant model doing every job. it writes the email. it picks the next tool. it decides if a source is relevant. it decides if a command is safe. every one of those decisions is a pick, a score or a yes/no, and each one costs a full generation call and seconds of waiting. the agent gets slower, the bill gets bigger, and at some point the loop costs more than the task is worth. this setup splits the work into three parts. the LLM writes. drafts, code, summaries, only the steps that produce text. JEV decides. which worker runs next, is this source relevant, is this tool call safe, is the draft ready. it never writes a word. it returns a choice, a score or a probability, with a confidence attached. CODE acts. hard rules like stop after ten actions, never publish without approval. anything below the confidence threshold goes back to the big model. the cheap layer handles the volume. the confidence score routes the hard cases. the frontier model only sees what actually needs thinking. every decision gets logged, and every log helps tune the threshold, so the agent gets cheaper and more accurate the longer it runs. public numbers from the demos: 1,018 research papers classified for 8 cents. 500 emails sorted for 3.5 cents. a claude session trimmed from nearly 1M tokens to 86K in about a second. most people are scaling the model. split the work. save this, and follow for more setups like this
This post quotes another post: x.com/imryven/status/2104255840819491030