105 planted bugs test: Astra fixes 45, Grok 4.7 fixes 28.7

Paweł Huryn @PawelHurynOriginally posted 2026.09.22

Open original on X

So I tested Grok 4.7 on real work. 2 repos. 105 planted bugs. Find and fix what you can. The numbers after 3 runs: GPT-6 Astra (max): 45 Muse Spark 1.3 (max): 32.2 Grok 4.7 (xhigh): 28.7 Grok 4.6 (xhigh): 28.7 (not a typo) Opus 5 (max): 27 Qwen3.8-Max (max): 25.7 Hate doing this, but won't lie. That's not the model we've been waiting for. Scores for more effort levels and n=5 to decide which Grok (4.6 vs. 4.7) wins drop in this thread every 30-70 min 🧵