Fine-tuning a Jev-style decision model locally with Unsloth
Unsloth can fine-tune Qwen, Llama and Gemma into models that make bounded decisions, with a 0.8B model reaching 78% holdout accuracy on about 4GB VRAM, and the author used one as a review layer for a data analysis agent.
Train your own Decision model like Jev locally! Unsloth now lets you fine-tune models like Qwen, Llama, and Gemma into decision models that make bounded decisions instead of generating text. The model reads the input together with the questions and options you define, then a small decision head scores those options and returns the selected answer with probabilities. So instead of using a generative model for every routing, classification, policy, or evaluation decision, you can train a much smaller model specifically for the decisions your system needs to make. Training is done with LoRA, and the dataset format is simple: input state, the questions to decide, and the gold answers. Unsloth also calibrates the model on held-out data, so the returned probabilities are meant to better reflect how often the model is actually correct. The reported results are pretty interesting. Qwen3.5-0.8B reached 78% holdout accuracy using around 4GB VRAM, while Llama 3.2 3B reached 79% at around 4.1GB VRAM. Qwen3.5-0.8B also improved from 36% to 73% on typed decisions after fine-tuning. Key capabilities: - Fine-tune Qwen, Llama, Gemma, or other supported LLMs into decision models - Train locally with 4-bit LoRA and relatively low VRAM - Handle choice, yes/no, and scored decisions in the same model - Return probabilities for each decision instead of generating free-form text - Calibrate the trained model on held-out data before serving it The part I like here is that you no longer need a purpose-built decision model to experiment with this architecture. You can take an existing open model, fine-tune it for your own decision space, and run the whole thing locally. I’ve shared the full Unsloth guide in the comments! I also built a data analyst agent with Jev as the review layer, and I’ve shared the full breakdown in the article below.
This post quotes another post: x.com/sumanth_077/status/2107473852850909566