Integrating Jev into an investment product: four use cases and pitfalls
An investment professional shares day-one notes on integrating Jev: prompt risk classification works well, while model routing, long-document pre-reading and skill matching hit snags.
Some nerdy Jev stuff for 10 of you that know what this is I've been trying to integrate this into my product. Some notes - quality is generally high and what is promised. Good intelligence. Promotable / steerable. Makes choices that align with much bigger models. Fast. Cheap. - but integrating it brings up a lot of nuanced challenges I didn't foresee some examples - easiest - categorizing incoming prompts on cyber / exfiltration risk. No downstream consequences to think about. Simple yes or no. Can now run it on every prompt. Nice. - model router - at first I thought I could use it to classify prompt difficulty and route to the right model. But I use codex, and changing from Luna to sol to Astra results in warning saying quality may suffer. Need to run testing. - document pre reader - investment work involves hunting for different in a lot of super long docs. Passing it all through Jev seem like an open and shut case. But for SOME reason, despite Jev returning all the right pages, the end result suffers. Detail gets dropped. My current hypothesis is that truncated docs somehow change what the LLM does. Investigating. - finding right skills / guides - way too many false positives, but not in a bad way. Like the probabilities that get returned are relatively similar. But it just doesn't cut enough to one shot the decision. Still need the model to reason over it. These are day 1 notes. Still a lot of things to test, optimize etc. Also I haven't even done any really cool stuff just yet.