有人质疑 Jev 重排 benchmark，作者逐条反驳
一位从业者指出网上那份用 80 条 SciFact 查询、ndcg@10 得出的 Jev 重排对比结论不成立，并列出七点理由，最后仍认为 Jev 大概率是 SOTA 的 pointwise 重排器。
Unfortunately almost everything about this tweet is a mix of incorrect or inconclusive >80 SciFact queries >ndcg@10 1. 80 queries is nowhere near enough to draw a statistically significant conclusion, esp. about reranking where performance differences between models aren't typically that large 2. ndcg@10 is an irrelevant metric in 2026 3. latency likely varies significantly by location or current Jev workload, I've also seen latency stats where Jev outperformed other rerankers like cohere 4 fast and voyage rerank 2.5 on p50 and p95 latency 4. SciFact queries are short factual statements, dissimilar to common query shapes you'd see from a search agent 5. SciFact's corpus has 5,183 documents in it, so there aren't going to be as many hard negatives for a given query to test reranking capabilities on 6. It's probably worth running the latest SOTA llms over scientific search tasks like this to see where they disagree with the existing ground truth on an IR dataset from 2020 with human labels to determine whether or not the ground truth needs fixing 7. Baseline problems - customers in the US and EU probably aren't comparing to a closed source qwen reranker you can only access via Alibaba's cloud offering at the moment. Most of them are likely unaware this new qwen reranker exists and it's unlikely its competitive with voyage rerank 3 for example Please be careful assuming the benchmarks/evals you see online like this apply to your workload. I'm still convinced it's more likely than not that Jev is an SOTA pointwise reranker when you measure it more accurately
这条是引用着另一条写的：「Milvus 实测 Jev 替代 RAG 重排器」