Jev routing results on Helix travel intents: 4x faster, 96-100% accuracy when confident
A travel AI team tested Jev on hundreds of real user messages for intent routing: accuracy on par with their small models, about 0.25s per decision (roughly 4x faster), lower cost, and 96-100% correct when confidence was 0.8 or higher.
Congrats @typesafeai team... tested Jev on some @HeyHelixAI core routing functions (for example "Is this a flight / hotel / restaurant / experience request?") against 100s of evals. Early test results: - Accuracy on par with, or slightly ahead of, the (already small, fast, cheap) LLMs we use today, on real user messages the models had never seen ✅ - Roughly 4x faster: about a quarter of a second per decision, versus 1 to 1.7 seconds (note some of this was network latency from Europe - US) ⚡ - A fraction of the cost per call 💷 - Confidence that means something: when it said it was confident (0.8+), it was right 96 to 100% of the time 🎯 - Matching remembered traveller preferences to hotels ("does this hotel have what I like?"): it found 97.6% of real matches, with zero false positives 🏨 Headline image from https://typesafe.ai - go check it out