Milvus tests Jev as a RAG reranker replacement
On 80 SciFact queries, Jev raised nDCG@10 by 0.0778 over no reranking, more than qwen3.7 rerank, but with 10.2x higher latency and 6.7x higher cost.
𝗖𝗮𝗻 𝗝𝗲𝘃 𝗿𝗲𝗽𝗹𝗮𝗰𝗲 𝗮 𝗿𝗲𝗿𝗮𝗻𝗸𝗲𝗿 𝗶𝗻 𝗥𝗔𝗚? We tested three setups on the same Milvus shortlist: 𝗻𝗼 𝗿𝗲𝗿𝗮𝗻𝗸𝗶𝗻𝗴, 𝗾𝘄𝗲𝗻𝟯.𝟳-𝘁𝗲𝘅𝘁-𝗿𝗲𝗿𝗮𝗻𝗸, 𝗮𝗻𝗱 𝗝𝗲𝘃. On 80 SciFact queries, qwen3.7-text-rerank improved nDCG@10 by 𝟬.𝟬𝟰𝟰𝟲 over no reranking, while Jev improved it by 𝟬.𝟬𝟳𝟳𝟴. Jev ranked best of the three, but its P50 reranking latency was 𝟭𝟬.𝟮× 𝗵𝗶𝗴𝗵𝗲𝗿 𝘁𝗵𝗮𝗻 𝗾𝘄𝗲𝗻, with an estimated cost per run of 𝟲.𝟳×. Why does Jev behave differently? Its interface is built around 𝘀𝘁𝗮𝘁𝗲 + 𝗾𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀. For reranking, we keep the relevance question fixed and change the state for each query-candidate pair: “𝗜𝘀 𝘁𝗵𝗶𝘀 𝗰𝗮𝗻𝗱𝗶𝗱𝗮𝘁𝗲 𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝘁𝗼 𝘁𝗵𝗲 𝗾𝘂𝗲𝗿𝘆?” Jev returns a structured Yes/No judgment with probabilities, and we use 𝗣(𝗬𝗲𝘀) as the relevance score for sorting. In this experiment, 30 candidates meant 30 independent Jev requests running concurrently, so the latency includes that API overhead. We also tested using probability directly for filtering. At a 𝟬.𝟱 𝘁𝗵𝗿𝗲𝘀𝗵𝗼𝗹𝗱, top-5 precision reached 𝟰𝟵.𝟰%, but 𝟭𝟴.𝟴% 𝗼𝗳 𝗴𝗼𝗹𝗱-𝗿𝗲𝗹𝗲𝘃𝗮𝗻𝘁 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀 𝘄𝗲𝗿𝗲 𝗳𝗶𝗹𝘁𝗲𝗿𝗲𝗱 𝗼𝘂𝘁. That makes Jev attractive for offline retrieval, data cleaning, and evaluation, but much harder to use as-is for latency-sensitive online RAG. Next, we’re testing whether batching multiple candidate questions into one Jev request can bring that latency down.