GPT-6 Sol bug-fixing test: lower score but ten times cheaper

Paweł Huryn @PawelHurynOriginally posted 2026.09.23

Open original on X

I finally tested GPT-6 Sol on a real work. 2 repos. 105 hidden bugs. Find and fixed what you can. It looks like a huge degradation. The results: - GPT-6 Astra (max): 45 - GPT-5.6 Sol (max): 43.5 - Opus 5.5 (max): 41.7 - Muse Spark 1.3 (max): 32.2 - GPT-6 Sol (max): 29.3 Until you measure the cost (API-equivalent): - GPT-6 Astra (max): $33.04 - GPT-5.6 Sol (max): $95.25 - Opus 5.5 (max): $58.53 - Muse Spark 1.3 (max): $18.11 - GPT-6 Sol (max): $9.93 More effort levels (xhigh, high, medium, low) dropping in this thread today 🧵