Opus 5.5 写 Blender 脚本:排行榜高分却在 4.2/5.2 版本上翻车
作者用 Opus 5.5 为 Blender 4.2 渲染农场写 Python,11 次调用有 7 次静默跑偏,指出榜单没测运行时版本。
Opus 5.5 sits at the top of the Artificial Analysis index, 58 against 53 for the next two models. It still writes Blender 4.2 Python for a 5.2 render farm, and the farm never complains. This isn’t a capability gap. The index runs ten evals: office work, terminal tasks, science code, long context, hard reasoning. None of them asks which build of a library is installed on the machine that runs the code. Where the score and the scene stop agreeing: → it leads six of the ten, and not one of them pins a version to the runtime → at max effort it spends about 119k output tokens per task, all of it careful reasoning inside the API it remembers → the runtime absorbs most stale calls instead of raising, so the loop never gets an error to correct against → a coherent script for the wrong release reads as competent to anyone not diffing it against the live build → run the same file on 4.2 and 5.2 and plot the outputs against each other: the engine line sits on the diagonal, the node and pass lines fall off it In this run, 7 of 11 drifted calls went through without a word. The last log line before the next frame came from the human reviewer: looks fine to me. The leaderboard measured the thinking. The farm needed a date. You don’t need a smarter model here, you need one that’s been told which build it’s standing on. The full desk, with every call sorted by how quietly it broke, is below 👇
这条是引用着另一条写的:x.com/0xcodio/status/2114351428433011132