Rippling tests Jev on classification in GrowthOS
Rippling tested Jev on anonymized real-world data in GrowthOS: it reached 90% classification accuracy versus a 50% LLM baseline and ran nearly 8x faster, but performed poorly as a control mechanism in its text-to-SQL agent loop.
We tested JEV in GrowthOS, Rippling’s agent harness for GTM, using anonymized real-world data. E.g. a marketer or RevOps user could ask, “Classify these campaign emails by strategy,” then analyze which messaging patterns perform best. JEV achieved 90% accuracy versus 50% for the LLM baseline and ran nearly 8x faster. It was not a universal win. JEV performed poorly as a control mechanism inside our text-to-SQL agent loop because it lacked the semantic context needed to interpret internal business definitions. JEV could see that a query returned rows, but not know whether those rows represented Rippling’s canonical definition of “qualified pipeline.” The LLM could inspect the relevant models, filters, and business definitions before deciding whether the analysis was complete. The takeaway: JEV looks extremely promising for repeatable 'classification' like workloads, while LLMs remain better suited to context-heavy reasoning. But pairing them is pretty freakin awesome.