jevPOST

Jev vs Gemini 3.8 Flash: labeling 1,000 app reviews

On the same data, batches and concurrency, Jev labeled 1,000 app reviews in 4.6s for $0.023 versus Gemini 3.8 Flash's 18.8s and $0.158, with 84% topic agreement and 93% bug-call agreement.

Vaishnavi

@_vmlopsArchived 2026.09.20Originally posted 2026.09.19

Vaishnavi @_vmlopsOriginally posted 2026.09.19

Open original on X

Someone raced Jev against Gemini 3.8 Flash, labelling 1,000 real app reviews. Same data, same batches, same concurrency Jev: 4.6s, $0.023 Gemini 3.8 Flash: 18.8s, $0.158 4.1× faster. 7× cheaper. Quality gap? Basically none 84% topic agreement, 93% bug-call agreement, sentiment correlation 0.80 vs 0.82 Jev used 8× more input tokens (every question repeats its rubric) but still crushed on cost, because output tokens are free Fully open, zero dependencies, recorded runs you can replay for free no API key needed

x.com/_vmlops/status/2101247228668969107Archived 2026.09.20 11:14