jevPOST
Jev scores about 19.7% on HLE multiple-choice set
On the same 513 text-only questions used in the AA Decision Index, the author measured Jev at 19.7% and compared it with a longest-answer heuristic and random guessing.
antoine_chaffin
@antoine_chaffinArchived 16 days agoOriginally posted 2026.10.14
antoine_chaffin @antoine_chaffinOriginally posted 2026.10.14
On HLE's 513 text-only multiple-choice questions (the set used in the AA Decision Index), I measured Jev at 19.7% (that's a bit below the AA chart's 21: setups differ slightly and its outputs aren't deterministic) On this exact set, always picking the longest answer scores 20.7%, vs 16.4% for random guessing That's above every decision model tested (the effect comes mostly from Math and Physics questions)
x.com/antoine_chaffin/status/2110335916170416189Archived 2026.10.15 10:43