Paper: plugging Jev decision models into an agent harness
The paper introduces JevAgentHarness, which routes routine choices to a distilled 4B decision model and falls back to the LLM below a fitted confidence threshold, cutting tool-cascade cost 14.2% with no outcome change across 500 tasks.
Must-read paper integrating decision models like Jev into harnesses. They distilled an open 4B decision model, mostly from Jev's answers on the agent's own states, that beats Jev inside that agent and is much cheaper to run. Their harness, JevAgentHarness, sends each routine choice to the decision model and uses its answer only when the confidence clears a threshold fitted on the agent's own trajectories. Below the threshold, the LLM decides as usual. With the fitted threshold, tool and argument cascades cut cost by 14.2% without changing the outcome of any of 500 tasks. A fixed threshold of 0.5 lost 21.5 points. They also train AgentDecision-4B on states the agent reaches after its own mistakes, with Jev's answers as labels. On tasks unlike its training data, the agent solved 91.5% with it and 82.0% with Jev. On familiar tasks, 4B models trained this way made agents 81.1 to 89.0% cheaper and up to 2.5 times faster while staying within 3 points of success. Paper: https://academy.dair.ai/papers/jevagentharness-benchmarking-deploying-and-training-decision-models-that-share-c-2610.21412