AnyJev turns open LLMs into Jev-style decision models
Nokia Applied Research open-sourced AnyJev, a Python library that extracts typed choices, yes/no judgments, scores, and probabilities from causal LLMs, with L0/L1/L2 calibration levels.
Turn any open LLM into a Jev-style model: Researchers just open-sourced AnyJev, a Python library that extracts typed choices, yes/no judgments, scores, and probabilities from causal LLMs. Beyond its raw baseline, AnyJev offers three levels depending on how much labeled data and calibration you need: - L0 requires no labels. It rotates the answer options to reduce position bias, but its probabilities remain uncalibrated. - L1 uses 100-500 labeled examples per question to calibrate those probabilities through temperature scaling. It improves confidence estimates without changing the answer ranking. - L2 uses 100-300 labels to fit a closed-form head on an intermediate hidden state. It leaves model weights untouched and can stop inference before the full forward pass completes. The main constraint is model access. AnyJev requires access to logits or hidden states, so it is built for open models. L2 heads are also specific to a model and question. It is compatible with several open causal LLMs. Repo: https://github.com/nokia-applied-research/AnyJev (don't forget to star it ⭐ ) If you want to dive deeper, I also wrote about a similar mechanism in a hands-on guide. It covers building a Jev-style decision path with open models, entirely locally. Read it below.
This post quotes another post: x.com/1175166450832687104/status/210156361064449…