同一 V4.1 Flash 在 9 个 harness 跑分差 8.6 分

kiruwaaaa @kiruwaaaaaa原推发布于 2026.10.22

在 X 上打开原推

holy sh*t, the same model scored 31.8 in one tool and 40.4 in another everyone is asking if opus 5.5 got nerfed. a 754k-view post says yes, another one reran a single prompt 20 days apart and posted the difference before you trust a screenshot, look at this test > one model: deepseek v4.1 flash 1017 > 9 harnesses, 5 attempts each, 89 runs in total > pi harness and grok build cli: 40.4 > claude code: 36.4 > kilo code: 31.8 same weights, 8.6 points apart. that gap came from the wrapper, not the model the vendor's own harness added 2-3 points and cost about 80% more, because it spawns more subagents and writes more output what actually got me is how most of us test. one prompt, one run, one screenshot, then "nerfed" how to check if your model really got worse > pin the exact model version > keep the harness, effort and tools identical > run the same 10 tasks, 5 attempts each > compare the spread, not a single output > log tokens per task. if your memory files and tools grew, usage grows with them if the numbers still move with everything pinned, then you have a story save this before the next "nerfed" thread which harness are you running right now? full stack in the article below

这条是引用着另一条写的：x.com/kiruwaaaaaa/status/2052447866208047296