实测 Jev API：只能做分类和置信度预测
作者直播一小时实测 Typesafe Jev API，指出它不能生成新 token，必须先程序化给出候选列表，输出固定为 Choice、Noul、Score 三类。
Spent an hour on YT livestream today with the Jev model via the typesafe API. Thanks to typesafe team for helping out with credits. The main use-case for this LM is to predict context aware decisions when the decision space of the problem is known apriori. The biggest downside is that you CANNOT generate new tokens - it only works as a classifier + confidence-predictor. You cannot use it even for extraction or form-filling stuff. You always have to pass in a list of candidate answers as well. IOW, you must programmatically create a candidate list BEFORE calling the API - and the model will output the probabilities for each entry given the context. You can send few shot examples and instructions too... dspy-style... but the output schema is fixed into 3 types of things: Choice (enums), Noul (boolean+probability), and Score (ordered value) We also went into masked LM and parallel constrained decoding. We discussed theories about how the model functions - and how it happens so quickly. Random thought: I feel like the next version of this model is probably going to be running small autoregressive/diffusion heads over the output token embeddings. This way prefill can still remain parallel, but arbitrary token output is also possible.
这条是引用着另一条写的：「上手 Jev API：像 DSPy 一样声明输出结构」