拆解 Jev 的 confidence 分数:只是分类概率的派生值
作者查文档和 demo 后发现 Jev 返回的置信度并非贝叶斯式不确定性,而是由最大概率算出的派生指标,不含额外信息,模型仍可能高置信地选错。
I was super intrigued when I noticed that JEV returns not just probs for candidate choices, but also a CONFIDENCE SCORE of it's prediction... I was wondering if TypeSafe AI brought back Bayesian Networks or something to learn uncertainty. Turns out - NO. Acc to their docs, confidence is just a derived metric from the classification scores. It carries ZERO extra information. Their demo page shows: confidence = (3 × largest probability − 1) / 2 Confidence is tightly coupled with the classification scores it is measuring the uncertainty for. There is no magic here. The model can produce high probability on a wrong choice and just be confidently wrong.