LangChain wires Jev into its agent loop for three jobs
LangChain showed Jev judging task difficulty, blocking risky tool calls at runtime, and grading agent answers against rubrics in online evals, with Jev answering PII detection instantly at 98% accuracy.
LangChain put Jev inside their own agent loop and showed the three jobs they hand it In the demo an LLM took about 5 seconds to say whether a text contains PII, and Jev answered immediately with 98% So their coding agents can ask Jev how hard a task is and save the powerful model for the hard ones Their auto mode middleware asks Jev whether a tool call is risky and blocks it at runtime, a check she had switched off because it used to be too slow And the same model grades an agent's answer against a rubric in online evals: correct, matches the reference, grounded, cites a source My guide points that setup at research sources, with a 0 to 3 relevance rubric over 50 papers and a review queue for the low-confidence calls It also checks which useful papers the filter quietly archived Watch the session, then grab the starter and the prompts below
This post quotes another post: x.com/alphabatcher/status/2101725422517997896