Chroma CEO on LLM reranking, plus a Jev build guide
Chroma CEO Jeff Huber explains how to use an LLM to narrow 10,000 candidate chunks down to the 20 that matter, with a guide to building that screening layer with Jev and testing it on 50 papers.
Chroma CEO Jeff Huber: "Using an LLM as a reranker and brute forcing from 300 down to 30, I've seen now emerge a lot" 57 minutes on picking what an LLM reads, from 10,000 candidate chunks down to the 20 that matter He's heard of teams running their own models for about a penny per million input tokens, with output cost near zero A research agent has the same problem with 50 papers: every weak match competes with the ones it needs Watch it, then use the guide below to build that screen with Jev and test it on 50 papers you label yourself
This post quotes another post: x.com/0xtreff/status/2101722231759941805