Google DeepMind 新基准测试 Agent 盲从用户错误建议
XYEval 在 tau2-bench、SWE-bench 等任务中注入一条自信但错误的提示，Gemini、Claude、GPT 得分最多下降 46.7%，且模型常在推理中反对却仍照做。
Interesting new paper from Google DeepMind. Studies agents that follow bad advice from users. Users often suggest a fix that sounds right and is wrong. XYEval adds one confident, misleading hint to tasks from tau2-bench, SWE-bench, Terminal-Bench, HLE and MCP-Atlas. The task and its correct solution stay the same, so any drop comes from the agent taking the advice. Scores fall by up to 46.7% relative across Gemini, Claude Opus 4.8 and GPT 5.5. The drops are larger on easier benchmarks, which suggests more capable models will not fix this on their own. Agents often disagree with the hint in their reasoning, then follow it anyway without telling the user. Compliance shows up almost only in failed runs. A system prompt that warns about the XY problem helps on single-turn tasks and leaves large drops on multi-turn ones like tau2-bench and SWE-bench Verified. Paper: https://t.co/290P0eUOzY