Rippling 在 GrowthOS 实测 Jev 分类任务
Rippling 用真实数据测 Jev，分类准确率 90% 对基线 50% 且快近 8 倍，但在 text-to-SQL 控制环节表现差。
We tested JEV in GrowthOS, Rippling’s agent harness for GTM, using anonymized real-world data. E.g. a marketer or RevOps user could ask, “Classify these campaign emails by strategy,” then analyze which messaging patterns perform best. JEV achieved 90% accuracy versus 50% for the LLM baseline and ran nearly 8x faster. It was not a universal win. JEV performed poorly as a control mechanism inside our text-to-SQL agent loop because it lacked the semantic context needed to interpret internal business definitions. JEV could see that a query returned rows, but not know whether those rows represented Rippling’s canonical definition of “qualified pipeline.” The LLM could inspect the relevant models, filters, and business definitions before deciding whether the analysis was complete. The takeaway: JEV looks extremely promising for repeatable 'classification' like workloads, while LLMs remain better suited to context-heavy reasoning. But pairing them is pretty freakin awesome.