SEAR benchmark tests self-evolving robot agents across 7 frontier models
SEAR is a new benchmark and framework for evaluating how LLM agents self-evolve on robots, testing 7 frontier models on 330 simulation tasks, 2 physics engines and real hardware; Astra did best at watching and correcting moves live, while Fable did best at writing its own code tools first.
Amazed by Astra controlling robots? It turns out other LLMs can do it too – even the text-only ones like DeepSeek. We also find Astra did best watching and correcting every move live; Fable did best building its own tools and policy in code first. We introduce SEAR, a new benchmark and framework to rigorously evaluate and study how LLM agents self evolve on robots. 🔁📈The loop: within a controlled time budget, an agent and its library update a policy → the policy runs in simulation or on hardware → a judge returns feedback → the library carries the skills and lessons into the next task. 💻🚏 Self-evolving agents operate in myriad ways - we study all of them at once: the policy can be code the agent wrote (code-as-policy), the agent itself in the control loop (agent-as-policy), or a VLA it trained (autoresearch). 🧑🔬🤖We spent 4+ years of accumulated agent evolving time across simulation and real hardware and tested 7 frontier models (GPT-6-Astra, Fable 5.1, Opus 5, GPT-5.6-sol, Gemini-3.6-flash, Kimi-k3, DeepSeek-v4-flash) on 330 sim tasks, 2 physics engines, and real robots with different embodiments. 🌐Project overview: http://sear.bot A thread 🧵1/5