Batching 10 questions into one Jev call cuts latency from 6.6s to 0.66s
The author tested merging 10 questions into a single Jev call, cutting latency from 6.6 seconds to 0.66 seconds and input tokens from 62k to 6.2k, and gives the rule: same state with no dependencies means the same call.
you're f*cking up jev and paying 10x for the exact same answers almost every agent calls it like an LLM: one question, one request, wait, repeat jev isn't an LLM. it takes one state and a stack of typed questions, and answers all of them at once the questions never see each other. so there's zero reason to ask them one by one the receipt: ↳ 10 questions, 10 calls: ~6,600 ms, ~62k input tokens ↳ 10 questions, 1 call: ~660 ms, ~6.2k input tokens ↳ same answers. identical // latency was: 10 round trips stacked back to back now: one round trip. the whole agent step lands under a second // spend was: the full state re-sent with every single question now: the state goes up once. at $0.042 per 1M input tokens that's $2,604 vs $260 per million agent steps // throughput was: 1,200 req/min limit ÷ 10 calls = 120 steps a minute now: 1,200 steps a minute. same key // the catch questions are blind to each other. "if billing, which plan?" is still a second call one budget per call: ~64k tokens across all questions jev can't count. one noul per item, sum it in code the rule: same state + no dependency = same call full drawing below: one angry ticket, 10 questions, both loops racing grep your agent for jev calls inside a for loop. tonight
This post quotes another post: x.com/n01ennn/status/2103542021071978601