jevIMAGE

Jev runs LLM-Wikirace: 450 games for under $1

Someone ran Jev on the LLM-Wikirace benchmark for 450 games over 8 hours at under $1 total, concluding it is fast and extremely cheap but weaker than frontier LLMs on world knowledge and planning.

Ilija Bogunovic

@ilijabogunovicArchived 2026.09.22Originally posted 2026.09.22

Jev runs LLM-Wikirace: 450 games for under $1

Ilija Bogunovic @ilijabogunovicOriginally posted 2026.09.22

Open original on X

@TypeSafeAI just released Jev, a new System 1 model, with Wikirace as a headline demo. So we ran it on ours: LLM-Wikirace benchmark. 450 games. 8 hours of wall-clock time. Under $1 total. Verdict: fast and absurdly cheap, but short on the world knowledge and planning that frontier LLMs bring. 🧵

x.com/ilijabogunovic/status/2102075332014624819Archived 2026.09.22 10:14