Building a Jev classifier with confidence-based fallback in Claude Code
A classification pipeline run in Claude Code: a frontier model creates labels, Jev classifies first and falls back to a stronger model below a confidence threshold, with prompts iterated in a loop.
VERY underrated classification AI flow: ALL DONE IN CLAUDE CODE: 1) Have fable create a prompt for classifying a thing 2) Then, have claude code run 100 samples through with fable as the model. Spot check these. These are source of truth. 3) Then, have claude code build the wiring to run it through jev first. 4) Have fable examine the confidence scores of jev. You're looking to see 'if confidence falls off <.7, should we fall back to the frontier?' 5) Have claude code string up a fall back to luna when confidence from jev is < x. 6) Evaluate the classifications jev/luna made using fable. 7) Have fable update the prompt to catch edge cases 8) Repeat until good You now have a low cost VERY accurate classifier for almost anything