Independent test: Jev's 200x speed claim falls to 3.6x, but the cascade is more useful
AY Automate retested with 791 labeled decisions and found Jev's median latency at 0.33s versus Terra's 1.17s, only 3.6x faster; routing the confident majority to Jev and the uncertain 19-23% to Terra matched accuracy at roughly 26-28% of the cost.
JEV'S 200× SPEED STORY COLLAPSED TO 3.6× IN AN INDEPENDENT TEST AND THE REAL RESULT WAS ACTUALLY MORE USEFUL TypeSafe reports 193.6× faster and 444.6× cheaper on its own workflow evaluation AY Automate tested 791 labeled decisions instead Jev's median latency was 0.33 seconds. GPT-5.6 Terra's was 1.17 seconds. That is 3.6×, not 193.6×. Jev also landed roughly five to six accuracy points below Terra on the two routing tasks Then they stopped treating it like a winner-takes-all model Jev handled the confident majority. Only the uncertain 19–23% went to Terra. The cascade matched or slightly beat Terra's measured accuracy while using roughly 26–28% of its estimated cost That is a much stronger product story than a giant benchmark multiple Cheap model first Expensive model only when uncertainty earns it Jev may matter most not as a replacement for frontier models, but as the gate that decides when they are worth paying for
This post quotes another post: x.com/fluixoo/status/2103123493567025321