Jev filters 40 billion data points every 15 minutes, replacing per-result LLM calls
A research team moved the LLM out of the hot path and put Jev in as a judge that scores data and decides what the system investigates next, cutting p95 latency at each stage.
JEV FILTERS 40 BILLION DATA POINTS EVERY 15 MINUTES. AN LLM ON EVERY RESULT COULDN'T DO IT a research team pushes that volume through their pipeline. the first version put an LLM on every result. it worked, and it was unusable: millions of explanations nobody read, latency stacking at every stage, inference bills growing with the dataset. so they moved the LLM out of the hot path and put a judge in it. jev doesn't explain the data. it scores it and decides what the system investigates next. this is the telemetry from that redesign. p95 latency before and after, stage by stage: planner -24ms, retriever -110ms, generate and verify pulled down with them. the flame strip at the top is a single agent step, 2,848 samples, and 27.6% of it is still sitting in read/poll. the pareto underneath is the part i keep coming back to. a handful of causes account for almost everything that fails. you don't fix those with a smarter model. you fix them with something cheap standing at the gate, deciding what gets through. i went through the whole decision layer. i'll show how the filtering actually works next.