Retagging 2,300 AI papers with Jev for $0.14
The author reclassified about 2,300 papers with Jev in 83 seconds for $0.14, found 579 high-confidence topic changes, and accepted all 30 disagreements he manually checked.
Found a great production use case for Jev. I used Jev to organize ~2.3K AI research papers. The total cost was $0.14, and it took about 83 seconds. The process: The papers already had old tags, which I ran through a previous open model (DeepSeek V4 Flash). However, I wasn't confident in the classifications, and I didn't want to spend more on tokens unless I spent time tuning it into a good LLM classifier via few-shot (more expensive). Too tedious, too costly, and unsustainable. Luckily for us, we now have Jev to help us with organizing papers better. So Jev first went through all the papers and retagged them. It agreed with 75% of the previous tags. Jev found about 579 high-confidence topic changes. I evaluated reliability by manually labeling 30 disagreements and accepted all of them. I was astonished by Jev's classification capabilities. We applied and verified all changes in production. I'm much more satisfied with the classifications, but I think there is still room for improvement. Check out the papers here: https://academy.dair.ai/papers I will experiment with Jev more. The takeaway is that pipelines can be significantly improved by carefully combining System One and System Two models. Jev clearly unlocks more interesting ways to organize papers and offer a more useful discovery layer for research papers. More updates on that soon.