gpt-6IMAGE
BrokenArXiv and ArXivMath benchmarks updated with recently refuted conjectures
The new benchmark versions use conjectures refuted on ArXiv in the past month and run models inside a harness rather than via API, with GPT-6 Astra still ranking first.
Jasper Dekoninck
@j_dekoninckArchived 2026.09.17Originally posted 2026.09.16
Jasper Dekoninck @j_dekoninckOriginally posted 2026.09.16
We are releasing the latest version of BrokenArXiv and ArXivMath! These benchmarks now focus on conjectures that were refuted in the last month on ArXiv, and models are executed within a harness instead of directly via API. Performance remains impressive, with GPT-6 Astra on top
x.com/j_dekoninck/status/2100180792601420138Archived 2026.09.17 10:18