DAIR's emotion dataset used to test new classifier Julia-1
The author tested Julia-1 on an emotion dataset built at DAIR and urges agent harness builders to experiment with mixing System One models like Jev with System Two models.
Cool to see classification models resurge in the era of agents. The effects of Jev are insane. Also very cool to see our emotion dataset built at @dair_ai used to test this new model, Julia-1. I spent my entire PhD building efficient classifiers from scratch (from graph-based to deep learning), so it's exciting to see this trend again. So much nostalgia. More importantly, if you build agent harnesses today (where all the edge is now that model capabilities have clustered), it's worth spending time running experiments that combine System One models and System Two models. You'll realize how many problems a great classifier (what I now call a System One model) can solve. Hybrid systems have historically performed better on domain-specific problems, and that still holds, even with general-purpose frontier models. Build your harness, create the evals, and run the experiments yourself. You will learn so much and gain great insights to take advantage of these advancements. This is a great example of what open-source enables. This is a work in progress. The cloud GPU spending on Julia 1 training and experiments was about R$540 (US$104.08). Will we see a new type of frontier lab focused on these models?
This post quotes another post: x.com/supersonicai/status/2103922653585162248