gemini-4POST
Gemini 4 Argon's 21% coding benchmark score may reflect the runner, not the model
The author argues the score measures the model plus the benchmark runner, including tools, time limits and retries, so 21% does not necessarily mean the model is weak at coding.
Sebastian Buzdugan
@sebuzduganArchived 21 days agoOriginally posted 2026.10.15
Sebastian Buzdugan @sebuzduganOriginally posted 2026.10.15
how can gemini 4 argon score 21% on every's senior-engineer benchmark without proving it is a weak coding model? the score measures the model plus the benchmark runner: tools, time limits, and retries all change which tasks finish
x.com/sebuzdugan/status/2110620437596274712Archived 2026.10.16 10:02