ARC-AGI-3 test: Astra uses reasoning tokens in far fewer actions than Sol

Matt Mazur @mhmazurOriginally posted 2026.09.05

Open original on X

With low reasoning effort in our standard harness, GPT-5.6 Sol used reasoning tokens in 98% of the actions it took while playing ARC-AGI-3 public games. With the same low reasoning effort, GPT-6 Astra used reasoning tokens in only 27% of its actions (!). We saw a similar pattern with the semi-private games, suggesting this difference was not driven by public benchmark contamination and may instead reflect architectural changes in how the GPT models work.

This post quotes another post: x.com/mikeknoop/status/2095932949350994342