MLflow 3.17: Jev models work as LLM eval judges
Setting model="typesafe:/jev-latest" in make_judge() lets Jev return a structured decision that MLflow stores as an assessment with the probability in metadata, so yes/no evals don't need a general-purpose LLM on every row.
Yes/no evals don’t need a general-purpose LLM on every row. That gets slow and expensive at volume. In MLflow 3.17, TypeSafe’s Jev models work as judges. Keep your existing mlflow.genai.evaluate() flow and set model="typesafe:/jev-latest" on make_judge(). Jev returns a structured decision; MLflow stores it as an assessment and puts the probability in metadata, so close calls show up when you review. A versioned model ID keeps comparisons consistent across runs. 🔗 Learn more: https://mlflow.org/docs/latest/genai/eval-monitor/scorers/llm-judge/custom-judges/typesafe/ #MLflow #GenAI #LLMOps