Scoring and pruning 27k tests one by one with orchestrator and Jev
The author demonstrates a jevify workflow where an agent sets criteria, validates on a small sample, then scores and prunes 27,000 tests individually for about $2.
Some of you might have noticed already but, we have a new magic word: **jevify**! I've found the orchestrator+jev setup to be capable of creating some quite robust flows, especially where thoroughness is important, say w/ refactors. Below is an example where we individually evaluate and prune tautologies across 27k tests for ~$2. It comes down to an agent setting up the criteria, validating with a small sample set, and then applying across a larger data set & acting on the results. Now you can ofc do this with subagent orchestration too, but, they usually do not end up comprehensive, and you have to repeat-until-no-changes almost like a compiler pass, whereas here, we have individual scorings of every test, so it is complete in one step.