A day of wiring Jev into a recommender system, and its limits
The author got Jev working in the pbd recommender system but notes it can't do entity extraction, ranking features or nested decisions, limits the marketing doesn't mention.
After playing with the model for a day I have some thoughts. Completely agree with Dr Khattab here. There is no doubt one can build great software with JEV. If your product has a control-flow decision where the candidate actions are known beforehand - JEV is great. That said, this model/API is severely limited - no token extraction, no ranking features, no control over complex output shapes, no way to produce nested decisions. You can't extract entities from text, coz this model is NOT a free-form token producer. But the shit is so fast and cheap that one can excuse the shorcomings - provided the usecase fits and you can work around the API. The marketing does not mention any of that. And for good reason. I am testing it inside my recommender system for pbd - and it works! As long as you can programatically/statically construct a list of choices, you can pass a context string + these choices into JEV, and it will give you the probability to pick each choice. I think whats happening is that people who have been around deep learning can recognize what is going on under the hood with this model, and that these ideas have existed in the past as well. For years. Some people who got into AI through API calling and ChatGPT are seeing the magic of token-level classification for the first time. And this idea of "parallel token processing" is absurdly attractive to them. There is also the twitter cookie monsters who have jumped on the hype train and getting their cookies while the topic is still viral. I'm seeing some of the most nonsensical demos that have no real application in the real world beyond an eye-catching demo. They all go viral because they are indeed super impressive to look at. This is part of the problem for overselling what this model can do/what it actually is. Once you start incorporating the model into a real product, you will quickly realize what the strengths are, and what the limits are.
This post quotes another post: x.com/lateinteraction/status/2100787755928170552