ReFigBench：同一模型换外壳结果大不同
GPT-5.5 在 Claude Code 与 Codex 中跑同样 1000 个任务，成绩随外壳变化；基准要求把 arXiv 图表重建成可编辑 PPT，感知仍是主要瓶颈。
Impressive paper showing how much the harness changes a coding agent's results. Harnesses do play a huge role in what you are getting out of the models. GPT-5.5 was run inside both Claude Code and Codex on the same 1,000 tasks. With a specialized PowerPoint workflow, it improved inside one harness and got worse inside the other. The harness also changed scores when the prompt was identical. ReFigBench asks coding agents to rebuild real arXiv overview figures as editable PowerPoint slides that keep the text, layout and connections. It covers ten configurations across the GPT, Claude, MiMo and MiniMax families, scored by artifact checks, two families of LLM judges and blinded human comparisons. Perception is still the main bottleneck. The specialized workflow removed native connectors in every configuration, yet human judges still preferred its renderings in most matchups. Paper: https://t.co/5buGY1XGxM Chat with Paper: https://t.co/G16VcCG6Kf